Aug 04 09:08:02.753523 ip-10-0-136-20 systemd[1]: Starting Kubernetes Kubelet... Aug 04 09:08:03.158400 ip-10-0-136-20 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:08:03.158400 ip-10-0-136-20 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 09:08:03.158400 ip-10-0-136-20 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:08:03.158400 ip-10-0-136-20 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 09:08:03.158400 ip-10-0-136-20 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:08:03.160245 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.160161 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 09:08:03.165368 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165346 2562 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:08:03.165368 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165363 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:08:03.165368 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165367 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:08:03.165368 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165370 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:08:03.165368 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165374 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:08:03.165368 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165377 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165380 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165383 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165386 2562 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165389 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165392 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165395 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165398 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165401 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165404 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165406 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165409 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165412 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165414 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165417 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165419 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165422 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165425 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165428 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:08:03.165583 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165430 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165434 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165438 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165441 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165444 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165446 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165449 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165451 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165457 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165459 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165462 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165464 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165467 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165469 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165472 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165475 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165478 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165480 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165483 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165486 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:08:03.166077 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165489 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165491 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165494 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165497 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165499 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165501 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165504 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165507 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165510 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165512 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165514 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165517 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165519 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165522 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165525 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165528 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165530 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165533 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165536 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165538 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:08:03.166609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165541 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165543 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165546 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165549 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165551 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165554 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165557 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165562 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165566 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165569 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165572 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165576 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165579 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165582 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165585 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165588 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165591 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165594 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165597 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:08:03.167105 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165601 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165604 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165606 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165610 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165612 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165615 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165618 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165621 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165623 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165626 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165629 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165633 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165636 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165639 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165643 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.165646 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167141 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167150 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167154 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167157 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:08:03.167573 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167160 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167163 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167167 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167170 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167174 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167177 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167181 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167184 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167187 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167190 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167193 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167195 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167198 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167201 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167205 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167208 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167210 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167213 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167216 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167218 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:08:03.168050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167221 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167224 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167226 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167229 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167232 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167234 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167237 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167239 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167243 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167245 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167264 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167268 2562 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167271 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167274 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167276 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167279 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167282 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167284 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167287 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:08:03.168603 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167291 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167294 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167298 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167301 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167303 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167306 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167309 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167313 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167316 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167318 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167321 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167324 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167326 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167329 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167331 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167334 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167336 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167339 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167342 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167344 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:08:03.169069 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167347 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167349 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167352 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167354 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167357 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167359 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167362 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167365 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167368 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167370 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167373 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167375 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167378 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167381 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167384 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167386 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167389 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167391 2562 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167393 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167396 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:08:03.169568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167399 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167402 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167404 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167408 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167412 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167415 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167418 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167421 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167424 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167426 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167429 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167432 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167434 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167437 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167440 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167442 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167829 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167834 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167837 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:08:03.170054 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167841 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167844 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167847 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167849 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167852 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167855 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167859 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167861 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167864 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167867 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167869 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167872 2562 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167875 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167878 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167880 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167883 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167886 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167888 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167891 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167893 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:08:03.170559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167897 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167899 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167902 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167904 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167907 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167909 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167912 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167915 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167917 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167920 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167922 2562 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167925 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167927 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167931 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167934 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167937 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167940 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167943 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167946 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167949 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:08:03.171050 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167952 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167954 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167957 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167959 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167963 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167965 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167968 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167971 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167974 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167976 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167978 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167981 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167983 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167986 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167988 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167991 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167993 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167996 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.167998 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168002 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:08:03.171574 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168005 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168008 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168010 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168013 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168015 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168018 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168020 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168023 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168025 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168027 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168030 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168032 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168035 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168038 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168041 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168044 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168047 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168050 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168054 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:08:03.172079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168058 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168060 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168063 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168066 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168068 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168071 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168073 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168076 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168078 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168081 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168084 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168086 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168089 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168091 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168094 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168096 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168098 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168161 2562 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168168 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168174 2562 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168178 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 09:08:03.172557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168182 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168185 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168191 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168195 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168199 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168202 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168206 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168210 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168213 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168216 2562 flags.go:64] FLAG: --cgroup-root="" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168219 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168222 2562 flags.go:64] FLAG: --client-ca-file="" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168225 2562 flags.go:64] FLAG: --cloud-provider="external" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168228 2562 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168232 2562 flags.go:64] FLAG: --cluster-domain="" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168235 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168238 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168241 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168244 2562 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168262 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168266 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168269 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168272 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168275 2562 flags.go:64] FLAG: --contention-profiling="false" Aug 04 09:08:03.173088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168278 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168281 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168285 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168287 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168296 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168300 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168303 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168305 2562 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168308 2562 flags.go:64] FLAG: --enable-server="true" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168311 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168317 2562 flags.go:64] FLAG: --event-burst="100" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168321 2562 flags.go:64] FLAG: --event-qps="50" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168324 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168327 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168330 2562 flags.go:64] FLAG: --eviction-hard="" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168334 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168337 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168340 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168343 2562 flags.go:64] FLAG: --eviction-soft="" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168345 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168348 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168352 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168355 2562 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168358 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168361 2562 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 09:08:03.173699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168364 2562 flags.go:64] FLAG: --feature-gates="" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168367 2562 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168370 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168373 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168376 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168379 2562 flags.go:64] FLAG: --healthz-port="10248" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168382 2562 flags.go:64] FLAG: --help="false" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168385 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168388 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168391 2562 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168394 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168397 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168400 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168403 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168406 2562 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168409 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168411 2562 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168415 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168419 2562 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168422 2562 flags.go:64] FLAG: --kube-reserved="" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168426 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168429 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168433 2562 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168435 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 09:08:03.174342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168438 2562 flags.go:64] FLAG: --lock-file="" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168441 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168444 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168447 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168452 2562 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168455 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168458 2562 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168461 2562 flags.go:64] FLAG: --logging-format="text" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168464 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168467 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168470 2562 flags.go:64] FLAG: --manifest-url="" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168473 2562 flags.go:64] FLAG: --manifest-url-header="" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168477 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168480 2562 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168483 2562 flags.go:64] FLAG: --max-pods="110" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168486 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168489 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168492 2562 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168495 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168497 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168500 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168503 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168510 2562 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168513 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168516 2562 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 09:08:03.174934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168521 2562 flags.go:64] FLAG: --pod-cidr="" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168524 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168529 2562 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168532 2562 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168536 2562 flags.go:64] FLAG: --pods-per-core="0" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168540 2562 flags.go:64] FLAG: --port="10250" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168543 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168546 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-039e8887dee1b5fbf" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168549 2562 flags.go:64] FLAG: --qos-reserved="" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168552 2562 flags.go:64] FLAG: --read-only-port="10255" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168555 2562 flags.go:64] FLAG: --register-node="true" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168558 2562 flags.go:64] FLAG: --register-with-taints="" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168562 2562 flags.go:64] FLAG: --registry-burst="10" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168567 2562 flags.go:64] FLAG: --registry-qps="5" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168570 2562 flags.go:64] FLAG: --reserved-cpus="" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168573 2562 flags.go:64] FLAG: --reserved-memory="" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168577 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168580 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168583 2562 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168586 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168588 2562 flags.go:64] FLAG: --runonce="false" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168591 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168594 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168597 2562 flags.go:64] FLAG: --seccomp-default="false" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168600 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 09:08:03.175554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168602 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168605 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168609 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168612 2562 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168615 2562 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168617 2562 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168620 2562 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168625 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168628 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168631 2562 flags.go:64] FLAG: --system-cgroups="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168633 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168640 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168643 2562 flags.go:64] FLAG: --tls-cert-file="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168646 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168650 2562 flags.go:64] FLAG: --tls-min-version="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168653 2562 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168655 2562 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168658 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168661 2562 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168664 2562 flags.go:64] FLAG: --v="2" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168668 2562 flags.go:64] FLAG: --version="false" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168673 2562 flags.go:64] FLAG: --vmodule="" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168676 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.168679 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168813 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:08:03.176160 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168818 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168821 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168824 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168827 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168830 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168833 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168835 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168838 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168841 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168844 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168846 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168849 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168851 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168854 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168858 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168860 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168863 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168866 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168870 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168872 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:08:03.176829 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168875 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168877 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168880 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168883 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168885 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168888 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168890 2562 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168893 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168896 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168899 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168902 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168904 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168907 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168909 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168912 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168915 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168918 2562 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168920 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168923 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168925 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:08:03.177347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168928 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168930 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168933 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168935 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168938 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168940 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168944 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168946 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168949 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168952 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168956 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168958 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168961 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168963 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168966 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168969 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168971 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168974 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168976 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168979 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:08:03.177842 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168981 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168984 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168989 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168992 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168995 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.168998 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169001 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169003 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169006 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169008 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169011 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169013 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169016 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169020 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169022 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169025 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169029 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169033 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:08:03.178343 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169038 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169041 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169044 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169046 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169050 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169053 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169055 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169058 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169060 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169063 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169066 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169068 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169072 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169075 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169077 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169080 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169083 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169086 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169089 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:08:03.178786 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.169091 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.169096 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.175853 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.175868 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.176676 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.176684 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176741 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176746 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176749 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176752 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176755 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176758 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176761 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176764 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176767 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:08:03.179268 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176770 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176773 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176775 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176778 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176780 2562 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176783 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176785 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176788 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176790 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176793 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176796 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176799 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176801 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176804 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176806 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176810 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176812 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176815 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176818 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176821 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:08:03.179645 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176824 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176826 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176829 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176831 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176834 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176837 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176840 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176842 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176845 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176848 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176851 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176854 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176857 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176859 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176862 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176865 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176867 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176870 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176872 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:08:03.180126 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176875 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176877 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176880 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176883 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176885 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176888 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176891 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176893 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176896 2562 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176901 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176905 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176909 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176913 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176916 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176918 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176921 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176924 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176927 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176929 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:08:03.180609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176932 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176934 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176937 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176939 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176942 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176945 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176947 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176950 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176953 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176955 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176959 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176963 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176965 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176969 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176971 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176974 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176977 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176980 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176983 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:08:03.181086 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176985 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176988 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176991 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176993 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176996 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.176999 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177002 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177005 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177007 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177010 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177012 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177015 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177019 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.177025 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:08:03.181630 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177129 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177134 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177139 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177142 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177144 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177147 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177150 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177153 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177155 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177158 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177160 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177163 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177165 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177168 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177171 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177173 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177176 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177178 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177181 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177184 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:08:03.181979 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177186 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177189 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177191 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177194 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177197 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177200 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177202 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177204 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177207 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177209 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177212 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177214 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177217 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177219 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177222 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177224 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177227 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177229 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177232 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177234 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:08:03.182475 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177237 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177240 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177242 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177245 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177247 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177261 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177264 2562 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177267 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177269 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177272 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177275 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177278 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177280 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177283 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177286 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177288 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177292 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177294 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177297 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177299 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:08:03.182967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177302 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177305 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177307 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177309 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177312 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177315 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177317 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177320 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177322 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177325 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177327 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177330 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177332 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177335 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177337 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177341 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177344 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177347 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177349 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:08:03.183453 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177353 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177356 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177359 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177362 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177364 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177367 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177371 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177374 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177377 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177380 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177387 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177390 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177393 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177395 2562 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177398 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177400 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177403 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177405 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177408 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:08:03.183928 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:03.177410 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.177416 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.177519 2562 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.179244 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.180777 2562 server.go:1019] "Starting client certificate rotation" Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.180871 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.180909 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 09:08:03.184405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.183580 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 09:08:03.188015 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.187993 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 09:08:03.197018 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.197001 2562 log.go:25] "Validated CRI v1 runtime API" Aug 04 09:08:03.198989 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.198975 2562 log.go:25] "Validated CRI v1 image API" Aug 04 09:08:03.199824 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.199807 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 09:08:03.204200 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.204177 2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8b1bf0df-033d-444f-b8db-0056abbc5789:/dev/nvme0n1p4 dc4787af-d5bd-4ede-9e4e-917dc38d9c75:/dev/nvme0n1p3] Aug 04 09:08:03.204287 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.204200 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 09:08:03.209124 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.208995 2562 manager.go:216] Machine: {Timestamp:2026-08-04 09:08:03.207789496 +0000 UTC m=+0.339999020 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101460 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c6e722c66990fac5d06401021f48e SystemUUID:ec2c6e72-2c66-990f-ac5d-06401021f48e BootID:e9cf63aa-6f8e-4c0a-a6fb-9c2247faab31 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:47:ed:71:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:47:ed:71:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:7a:dd:18:4c:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 09:08:03.209124 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.209118 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 09:08:03.209245 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.209221 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 09:08:03.211736 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.211720 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 09:08:03.212735 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.212712 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 09:08:03.212870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.212738 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 09:08:03.213285 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.213275 2562 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 09:08:03.213321 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.213288 2562 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 09:08:03.213321 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.213295 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 09:08:03.214175 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.214164 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 09:08:03.214815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.214805 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 09:08:03.217806 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217795 2562 state_mem.go:36] "Initialized new in-memory state store" Aug 04 09:08:03.217913 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217904 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 09:08:03.217942 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217933 2562 kubelet.go:486] "Attempting to sync node with API server" Aug 04 09:08:03.217969 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217943 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 09:08:03.217969 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217958 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 09:08:03.217969 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217967 2562 kubelet.go:398] "Adding apiserver pod source" Aug 04 09:08:03.218047 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.217975 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 09:08:03.218325 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.218314 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 09:08:03.218366 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.218331 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 09:08:03.222211 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.222194 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 09:08:03.223790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.223778 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 09:08:03.223841 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.223808 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 09:08:03.225005 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.224968 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 09:08:03.225005 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.224990 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 09:08:03.225005 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225000 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225008 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225021 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225030 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225042 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225050 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225061 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225069 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225081 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 09:08:03.225096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.225089 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 09:08:03.226041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.226028 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 09:08:03.226076 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.226044 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 09:08:03.228551 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.228524 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 09:08:03.228614 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.228524 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 09:08:03.229373 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.229360 2562 server.go:1268] "Started kubelet" Aug 04 09:08:03.229872 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.229457 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 09:08:03.229994 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.229938 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 09:08:03.230080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.230065 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 09:08:03.230198 ip-10-0-136-20 systemd[1]: Started Kubernetes Kubelet. Aug 04 09:08:03.233634 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.233606 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 09:08:03.235564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.235535 2562 server.go:310] "Adding debug handlers to kubelet server" Aug 04 09:08:03.236532 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.236511 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 09:08:03.237034 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.236994 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 09:08:03.238408 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238394 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 09:08:03.238489 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238441 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 09:08:03.238489 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.238481 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.238870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238852 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 09:08:03.238870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238869 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 09:08:03.239041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238900 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 09:08:03.239041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238988 2562 reconstruct.go:97] "Volume reconstruction finished" Aug 04 09:08:03.239041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.238997 2562 reconciler.go:29] "Reconciler: start to sync state" Aug 04 09:08:03.239215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239198 2562 factory.go:55] Registering systemd factory Aug 04 09:08:03.239215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239217 2562 factory.go:223] Registration of the systemd container factory successfully Aug 04 09:08:03.239520 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239507 2562 factory.go:153] Registering CRI-O factory Aug 04 09:08:03.239596 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239524 2562 factory.go:223] Registration of the crio container factory successfully Aug 04 09:08:03.239596 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239571 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 09:08:03.239596 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239593 2562 factory.go:103] Registering Raw factory Aug 04 09:08:03.239726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239609 2562 manager.go:1196] Started watching for new ooms in manager Aug 04 09:08:03.239931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.239918 2562 manager.go:349] Starting recovery of all containers Aug 04 09:08:03.241010 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.240030 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-20.ec2.internal.18c88f0030845d56 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-20.ec2.internal,UID:ip-10-0-136-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-20.ec2.internal,},FirstTimestamp:2026-08-04 09:08:03.229154646 +0000 UTC m=+0.361364169,LastTimestamp:2026-08-04 09:08:03.229154646 +0000 UTC m=+0.361364169,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-20.ec2.internal,}" Aug 04 09:08:03.245922 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.245895 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 09:08:03.245922 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.245900 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 09:08:03.246157 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.246133 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 09:08:03.249543 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.249525 2562 manager.go:354] Recovery completed Aug 04 09:08:03.253791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.253780 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.255428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255409 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.255488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255439 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.255488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255449 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.255857 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255845 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 09:08:03.255924 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255861 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 09:08:03.255924 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255893 2562 state_mem.go:36] "Initialized new in-memory state store" Aug 04 09:08:03.255924 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255914 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.256015 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.255986 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7h5p" Aug 04 09:08:03.256932 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.256919 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.256978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.256940 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.257015 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.256949 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.257069 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.257058 2562 policy_none.go:49] "None policy: Start" Aug 04 09:08:03.257101 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.257076 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 09:08:03.257101 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.257090 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 09:08:03.257590 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.257516 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-20.ec2.internal.18c88f0032153f92 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-20.ec2.internal,UID:ip-10-0-136-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-20.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-20.ec2.internal,},FirstTimestamp:2026-08-04 09:08:03.255426962 +0000 UTC m=+0.387636488,LastTimestamp:2026-08-04 09:08:03.255426962 +0000 UTC m=+0.387636488,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-20.ec2.internal,}" Aug 04 09:08:03.258168 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.258157 2562 policy_none.go:47] "Start" Aug 04 09:08:03.263452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.263437 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7h5p" Aug 04 09:08:03.298948 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.298815 2562 manager.go:337] "Starting Device Plugin manager" Aug 04 09:08:03.299052 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.298993 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 09:08:03.299052 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.299007 2562 server.go:87] "Starting device plugin registration server" Aug 04 09:08:03.299202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.299190 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 09:08:03.299231 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.299203 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 09:08:03.299324 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.299303 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 09:08:03.299422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.299383 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 09:08:03.299422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.299392 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 09:08:03.299812 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.299794 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 09:08:03.299884 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.299848 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.400058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.400029 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.400997 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.400978 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.401110 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.401011 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.401110 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.401024 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.401110 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.401054 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.407716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.407699 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.407804 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.407721 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-20.ec2.internal\": node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.409565 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.409520 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 09:08:03.409565 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.409564 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 09:08:03.409684 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.409587 2562 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 09:08:03.409684 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.409627 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 09:08:03.412623 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.412607 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:08:03.474891 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.474867 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.511520 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.511480 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal"] Aug 04 09:08:03.511632 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.511558 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.512523 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.512506 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.512631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.512536 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.512631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.512547 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.514898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.514885 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.515038 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.515118 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515052 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.515650 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515629 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.515744 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515660 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.515744 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515671 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.515744 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515633 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.515744 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515738 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.515898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.515750 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.517950 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.517934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.518036 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.517959 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:08:03.518678 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.518660 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:08:03.518785 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.518692 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:08:03.518785 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.518704 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:08:03.540003 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.539979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8e896688688bfbf603f7c1859e47653-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"c8e896688688bfbf603f7c1859e47653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.540094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.540010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2c069c649a5d879b02b53f674cdd665d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-20.ec2.internal\" (UID: \"2c069c649a5d879b02b53f674cdd665d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.540094 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.540035 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-20.ec2.internal\" not found" node="ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.540094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.540035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c8e896688688bfbf603f7c1859e47653-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"c8e896688688bfbf603f7c1859e47653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.544859 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.544843 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-20.ec2.internal\" not found" node="ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.575880 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.575859 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.640600 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.640579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8e896688688bfbf603f7c1859e47653-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"c8e896688688bfbf603f7c1859e47653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.640682 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.640612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2c069c649a5d879b02b53f674cdd665d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-20.ec2.internal\" (UID: \"2c069c649a5d879b02b53f674cdd665d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.640682 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.640630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c8e896688688bfbf603f7c1859e47653-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"c8e896688688bfbf603f7c1859e47653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.640764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.640680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c8e896688688bfbf603f7c1859e47653-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"c8e896688688bfbf603f7c1859e47653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.640764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.640704 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2c069c649a5d879b02b53f674cdd665d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-20.ec2.internal\" (UID: \"2c069c649a5d879b02b53f674cdd665d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.640764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.640681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8e896688688bfbf603f7c1859e47653-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal\" (UID: \"c8e896688688bfbf603f7c1859e47653\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.676240 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.676173 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.778816 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.778771 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.842870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.842846 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.848225 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:03.847943 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Aug 04 09:08:03.879569 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.879544 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:03.982059 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:03.981990 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:04.082758 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:04.082726 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:04.125774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.125737 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:08:04.181124 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.181096 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 09:08:04.181561 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.181221 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 09:08:04.181561 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.181274 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 09:08:04.183420 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:04.183405 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:04.195514 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.195494 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:08:04.266699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.266618 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 09:03:04 +0000 UTC" deadline="2028-01-20 04:39:11.668334668 +0000 UTC" Aug 04 09:08:04.266699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.266648 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12811h31m7.401689249s" Aug 04 09:08:04.283643 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:04.283617 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-20.ec2.internal\" not found" Aug 04 09:08:04.375204 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.375178 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:08:04.439763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.439738 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" Aug 04 09:08:04.448570 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.448554 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 09:08:04.449585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.449569 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" Aug 04 09:08:04.463239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.463220 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 09:08:04.468119 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:04.468092 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e896688688bfbf603f7c1859e47653.slice/crio-3aeb7d3f9e23d082d3f22d392fee6db06b322e909c830622f32beb17001cffe4 WatchSource:0}: Error finding container 3aeb7d3f9e23d082d3f22d392fee6db06b322e909c830622f32beb17001cffe4: Status 404 returned error can't find the container with id 3aeb7d3f9e23d082d3f22d392fee6db06b322e909c830622f32beb17001cffe4 Aug 04 09:08:04.472322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:04.472306 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:08:05.219247 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.219218 2562 apiserver.go:52] "Watching apiserver" Aug 04 09:08:05.226826 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.226807 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 09:08:05.229453 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.228317 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-k5fdn","openshift-ovn-kubernetes/ovnkube-node-857rg","kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal","openshift-dns/node-resolver-sf84q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal","openshift-multus/network-metrics-daemon-t72c2","openshift-network-diagnostics/network-check-target-zcjs4","openshift-network-operator/iptables-alerter-w6wnq","kube-system/global-pull-secret-syncer-cnb48","kube-system/konnectivity-agent-22b68","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7","openshift-cluster-node-tuning-operator/tuned-74h6b","openshift-image-registry/node-ca-lhc2c","openshift-multus/multus-additional-cni-plugins-b4vcj"] Aug 04 09:08:05.231103 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.231081 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:05.231213 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.231166 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:05.233423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.233404 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.235921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.235773 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.235921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.235803 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.235921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.235821 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 09:08:05.235921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.235868 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tmmwr\"" Aug 04 09:08:05.235921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.235875 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.238118 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238086 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.238268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.238268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238267 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 09:08:05.238619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238605 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.238677 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238661 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 09:08:05.238730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238715 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gzj64\"" Aug 04 09:08:05.238769 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.238742 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 09:08:05.239610 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.239594 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 09:08:05.240299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.240281 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.240592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.240570 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ttsfc\"" Aug 04 09:08:05.240691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.240614 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.240691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.240634 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.243030 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.243007 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.243133 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.243070 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:05.243199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.243174 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bvcfm\"" Aug 04 09:08:05.243313 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.243298 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.243313 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.243310 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.243450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.243374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 09:08:05.245677 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.245652 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.245773 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.245720 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:05.248129 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.248106 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.248980 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.248960 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c069c649a5d879b02b53f674cdd665d.slice/crio-8940624707e095aa2087ce43138f1f9cc898d40602cc381a605b172d7fa3cf3d WatchSource:0}: Error finding container 8940624707e095aa2087ce43138f1f9cc898d40602cc381a605b172d7fa3cf3d: Status 404 returned error can't find the container with id 8940624707e095aa2087ce43138f1f9cc898d40602cc381a605b172d7fa3cf3d Aug 04 09:08:05.250842 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:05.250842 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250741 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-socket-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.250842 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-device-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.250842 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/264b1ddb-48ef-4337-87c8-b107b0fce91b-tmp-dir\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.250842 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe00b519-b049-40dc-a899-3ee10e24a4b4-iptables-alerter-script\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250848 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe00b519-b049-40dc-a899-3ee10e24a4b4-host-slash\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250856 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-system-cni-dir\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cnibin\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.250974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-os-release\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cni-binary-copy\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjczx\" (UniqueName: \"kubernetes.io/projected/fe00b519-b049-40dc-a899-3ee10e24a4b4-kube-api-access-zjczx\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-registration-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rprf\" (UniqueName: \"kubernetes.io/projected/efe78a30-6ac2-4a4f-80b8-166c970b7e62-kube-api-access-6rprf\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251132 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t27cm\" (UniqueName: \"kubernetes.io/projected/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-kube-api-access-t27cm\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251153 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.251173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh692\" (UniqueName: \"kubernetes.io/projected/264b1ddb-48ef-4337-87c8-b107b0fce91b-kube-api-access-hh692\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251217 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-sys-fs\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251252 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m88qh\"" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251390 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251419 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/264b1ddb-48ef-4337-87c8-b107b0fce91b-hosts-file\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.251791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.251504 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 09:08:05.253557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.253538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.254119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.254079 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 09:08:05.254266 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.254116 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wnz2q\"" Aug 04 09:08:05.254266 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.254147 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 09:08:05.254266 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.254229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.254925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.254902 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 09:08:05.255029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.254982 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.255096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.255021 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 09:08:05.255234 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.255216 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 09:08:05.257937 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.257916 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.258025 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.258006 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6klgn\"" Aug 04 09:08:05.258571 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.258550 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 09:08:05.258631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.258579 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 09:08:05.260215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.260201 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.260687 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.260670 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 09:08:05.261002 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.260985 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7ncv5\"" Aug 04 09:08:05.263466 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.263444 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 09:08:05.263585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.263489 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6f7sq\"" Aug 04 09:08:05.263847 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.263744 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:08:05.317681 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.317652 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lcksm" Aug 04 09:08:05.327974 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.327944 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lcksm" Aug 04 09:08:05.340298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.340277 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 09:08:05.351581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-serviceca\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.351662 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-socket-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.351662 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/264b1ddb-48ef-4337-87c8-b107b0fce91b-tmp-dir\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.351662 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-systemd-units\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.351662 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-run-netns\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-log-socket\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00853198-747d-4269-8b81-7081eb8ddf70-ovn-node-metrics-cert\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351733 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-socket-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cnibin\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-os-release\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cni-binary-copy\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cnibin\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.351817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-var-lib-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-cni-multus\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-kubelet\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351879 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-os-release\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bggfv\" (UniqueName: \"kubernetes.io/projected/661c18ec-bd75-4b19-ad4f-72500024e020-kube-api-access-bggfv\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/264b1ddb-48ef-4337-87c8-b107b0fce91b-tmp-dir\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-systemd\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh692\" (UniqueName: \"kubernetes.io/projected/264b1ddb-48ef-4337-87c8-b107b0fce91b-kube-api-access-hh692\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.351992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-cni-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-netns\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-conf-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-etc-kubernetes\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnjwh\" (UniqueName: \"kubernetes.io/projected/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-kube-api-access-dnjwh\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-kubelet-config\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-sys-fs\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352297 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cni-binary-copy\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-ovn\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352316 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-sys-fs\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352363 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb2aba-45f2-408b-9c0d-30a4e561dd9f-agent-certs\") pod \"konnectivity-agent-22b68\" (UID: \"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f\") " pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352427 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-cnibin\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe00b519-b049-40dc-a899-3ee10e24a4b4-host-slash\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-env-overrides\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s29fs\" (UniqueName: \"kubernetes.io/projected/00853198-747d-4269-8b81-7081eb8ddf70-kube-api-access-s29fs\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.352787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/661c18ec-bd75-4b19-ad4f-72500024e020-cni-binary-copy\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-k8s-cni-cncf-io\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/661c18ec-bd75-4b19-ad4f-72500024e020-multus-daemon-config\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe00b519-b049-40dc-a899-3ee10e24a4b4-host-slash\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-modprobe-d\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysconfig\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352639 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-system-cni-dir\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysctl-d\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysctl-conf\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-tuned\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-system-cni-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rprf\" (UniqueName: \"kubernetes.io/projected/efe78a30-6ac2-4a4f-80b8-166c970b7e62-kube-api-access-6rprf\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-node-log\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-cni-netd\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54gdg\" (UniqueName: \"kubernetes.io/projected/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-kube-api-access-54gdg\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352745 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-system-cni-dir\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-run\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.353496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-tmp\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-systemd\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-cni-bin\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-hostroot\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.352999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-multus-certs\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-sys\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-device-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe00b519-b049-40dc-a899-3ee10e24a4b4-iptables-alerter-script\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353131 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95flg\" (UniqueName: \"kubernetes.io/projected/9f2c9e63-ede8-4b44-9f40-6e8370313c84-kube-api-access-95flg\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353149 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-device-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-slash\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-lib-modules\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.354238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb2aba-45f2-408b-9c0d-30a4e561dd9f-konnectivity-ca\") pod \"konnectivity-agent-22b68\" (UID: \"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f\") " pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-socket-dir-parent\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-cni-bin\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t27cm\" (UniqueName: \"kubernetes.io/projected/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-kube-api-access-t27cm\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-run-ovn-kubernetes\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-kubelet\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353577 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-var-lib-kubelet\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjczx\" (UniqueName: \"kubernetes.io/projected/fe00b519-b049-40dc-a899-3ee10e24a4b4-kube-api-access-zjczx\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353629 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe00b519-b049-40dc-a899-3ee10e24a4b4-iptables-alerter-script\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353660 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-ovnkube-config\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-dbus\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-host\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-registration-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-host\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353834 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-os-release\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.354755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-registration-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-kubernetes\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/264b1ddb-48ef-4337-87c8-b107b0fce91b-hosts-file\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.353978 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/efe78a30-6ac2-4a4f-80b8-166c970b7e62-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.354004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-etc-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.354030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-ovnkube-script-lib\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.354039 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/264b1ddb-48ef-4337-87c8-b107b0fce91b-hosts-file\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.355235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.354347 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.358941 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.358919 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 09:08:05.359012 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.358996 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:05.359047 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.359020 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:05.359047 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.359029 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:05.359119 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.359111 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:05.859072887 +0000 UTC m=+2.991282422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:05.362280 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.362254 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh692\" (UniqueName: \"kubernetes.io/projected/264b1ddb-48ef-4337-87c8-b107b0fce91b-kube-api-access-hh692\") pod \"node-resolver-sf84q\" (UID: \"264b1ddb-48ef-4337-87c8-b107b0fce91b\") " pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.362375 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.362313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t27cm\" (UniqueName: \"kubernetes.io/projected/4f074728-5d3c-4424-a6f7-c9aa4cd6632e-kube-api-access-t27cm\") pod \"multus-additional-cni-plugins-b4vcj\" (UID: \"4f074728-5d3c-4424-a6f7-c9aa4cd6632e\") " pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.362375 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.362317 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rprf\" (UniqueName: \"kubernetes.io/projected/efe78a30-6ac2-4a4f-80b8-166c970b7e62-kube-api-access-6rprf\") pod \"aws-ebs-csi-driver-node-cbhk7\" (UID: \"efe78a30-6ac2-4a4f-80b8-166c970b7e62\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.362591 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.362570 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjczx\" (UniqueName: \"kubernetes.io/projected/fe00b519-b049-40dc-a899-3ee10e24a4b4-kube-api-access-zjczx\") pod \"iptables-alerter-w6wnq\" (UID: \"fe00b519-b049-40dc-a899-3ee10e24a4b4\") " pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.414460 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.414405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" event={"ID":"2c069c649a5d879b02b53f674cdd665d","Type":"ContainerStarted","Data":"8940624707e095aa2087ce43138f1f9cc898d40602cc381a605b172d7fa3cf3d"} Aug 04 09:08:05.415618 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.415590 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" event={"ID":"c8e896688688bfbf603f7c1859e47653","Type":"ContainerStarted","Data":"3aeb7d3f9e23d082d3f22d392fee6db06b322e909c830622f32beb17001cffe4"} Aug 04 09:08:05.455381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/661c18ec-bd75-4b19-ad4f-72500024e020-multus-daemon-config\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.455381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-modprobe-d\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysconfig\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysctl-d\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysctl-conf\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-tuned\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-system-cni-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455514 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-node-log\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455515 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysconfig\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-cni-netd\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455555 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-modprobe-d\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.455594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455597 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-node-log\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455600 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysctl-d\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455608 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-sysctl-conf\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455626 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-cni-netd\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54gdg\" (UniqueName: \"kubernetes.io/projected/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-kube-api-access-54gdg\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-run\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455697 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-tmp\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455729 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-system-cni-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-systemd\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-cni-bin\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-hostroot\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455830 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-run\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-multus-certs\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-sys\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455886 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-systemd\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455893 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-cni-bin\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.456095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95flg\" (UniqueName: \"kubernetes.io/projected/9f2c9e63-ede8-4b44-9f40-6e8370313c84-kube-api-access-95flg\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.457073 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455971 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.457073 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-slash\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.457180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-lib-modules\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.457180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.457391 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb2aba-45f2-408b-9c0d-30a4e561dd9f-konnectivity-ca\") pod \"konnectivity-agent-22b68\" (UID: \"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f\") " pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.457527 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-socket-dir-parent\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.457527 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-cni-bin\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.457527 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-run-ovn-kubernetes\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.457527 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-kubelet\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/661c18ec-bd75-4b19-ad4f-72500024e020-multus-daemon-config\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-var-lib-kubelet\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-ovnkube-config\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-dbus\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.455941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-sys\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-host\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.457730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-host\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-os-release\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-kubernetes\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-kubelet\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-etc-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-ovnkube-script-lib\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-var-lib-kubelet\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.458055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-serviceca\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.458529 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-serviceca\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.458529 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458520 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-tmp\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-ovnkube-config\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.458567 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458568 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-tuned\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-run-ovn-kubernetes\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.457777 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-host\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458640 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-dbus\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458650 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-slash\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.458709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458700 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-os-release\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458772 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-lib-modules\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.458651 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:08:05.958627137 +0000 UTC m=+3.090836676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.458791 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458824 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-socket-dir-parent\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458839 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-host\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.458882 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:05.958859163 +0000 UTC m=+3.091068690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-hostroot\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458879 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-cni-bin\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458907 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-multus-certs\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458932 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-kubernetes\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458972 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-etc-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.458984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-systemd-units\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459120 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-systemd-units\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-run-netns\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-log-socket\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00853198-747d-4269-8b81-7081eb8ddf70-ovn-node-metrics-cert\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-var-lib-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-run-netns\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-cni-multus\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-kubelet\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459411 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bggfv\" (UniqueName: \"kubernetes.io/projected/661c18ec-bd75-4b19-ad4f-72500024e020-kube-api-access-bggfv\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459411 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-kubelet\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-systemd\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-log-socket\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459486 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb2aba-45f2-408b-9c0d-30a4e561dd9f-konnectivity-ca\") pod \"konnectivity-agent-22b68\" (UID: \"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f\") " pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459504 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-var-lib-openvswitch\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-cni-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-netns\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-conf-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.459679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-etc-kubernetes\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnjwh\" (UniqueName: \"kubernetes.io/projected/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-kube-api-access-dnjwh\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459686 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-ovnkube-script-lib\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-kubelet-config\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459737 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-netns\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459757 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-conf-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-ovn\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459791 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-var-lib-cni-multus\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459801 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-etc-kubernetes\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb2aba-45f2-408b-9c0d-30a4e561dd9f-agent-certs\") pod \"konnectivity-agent-22b68\" (UID: \"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f\") " pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-cnibin\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459906 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-env-overrides\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459957 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00853198-747d-4269-8b81-7081eb8ddf70-run-ovn\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459851 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-kubelet-config\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.459959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s29fs\" (UniqueName: \"kubernetes.io/projected/00853198-747d-4269-8b81-7081eb8ddf70-kube-api-access-s29fs\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/661c18ec-bd75-4b19-ad4f-72500024e020-cni-binary-copy\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.460836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460033 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-cnibin\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.461818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-k8s-cni-cncf-io\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.461818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-etc-systemd\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.461818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460159 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-host-run-k8s-cni-cncf-io\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.461818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460428 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/661c18ec-bd75-4b19-ad4f-72500024e020-multus-cni-dir\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.461818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.460557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00853198-747d-4269-8b81-7081eb8ddf70-env-overrides\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.461818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.461284 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/661c18ec-bd75-4b19-ad4f-72500024e020-cni-binary-copy\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.463990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.463965 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00853198-747d-4269-8b81-7081eb8ddf70-ovn-node-metrics-cert\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.464997 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.464969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb2aba-45f2-408b-9c0d-30a4e561dd9f-agent-certs\") pod \"konnectivity-agent-22b68\" (UID: \"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f\") " pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.467285 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.467256 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54gdg\" (UniqueName: \"kubernetes.io/projected/7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7-kube-api-access-54gdg\") pod \"node-ca-lhc2c\" (UID: \"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7\") " pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.472557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.472502 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bggfv\" (UniqueName: \"kubernetes.io/projected/661c18ec-bd75-4b19-ad4f-72500024e020-kube-api-access-bggfv\") pod \"multus-k5fdn\" (UID: \"661c18ec-bd75-4b19-ad4f-72500024e020\") " pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.472840 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.472816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnjwh\" (UniqueName: \"kubernetes.io/projected/e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f-kube-api-access-dnjwh\") pod \"tuned-74h6b\" (UID: \"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f\") " pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.473754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.473730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95flg\" (UniqueName: \"kubernetes.io/projected/9f2c9e63-ede8-4b44-9f40-6e8370313c84-kube-api-access-95flg\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.473839 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.473730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s29fs\" (UniqueName: \"kubernetes.io/projected/00853198-747d-4269-8b81-7081eb8ddf70-kube-api-access-s29fs\") pod \"ovnkube-node-857rg\" (UID: \"00853198-747d-4269-8b81-7081eb8ddf70\") " pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.544834 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.544807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" Aug 04 09:08:05.545207 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.545189 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:08:05.551435 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.551413 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" Aug 04 09:08:05.560420 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.560400 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sf84q" Aug 04 09:08:05.567205 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.567185 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w6wnq" Aug 04 09:08:05.575073 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.575045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lhc2c" Aug 04 09:08:05.583009 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.582991 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:05.589812 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.589794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:05.596600 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.596580 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k5fdn" Aug 04 09:08:05.603514 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.603490 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-74h6b" Aug 04 09:08:05.780606 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.780575 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbdb2aba_45f2_408b_9c0d_30a4e561dd9f.slice/crio-5a37f5ec907f2d04c8275feb7c48db2d8b01cd76ebb60cc454f203558f502ef2 WatchSource:0}: Error finding container 5a37f5ec907f2d04c8275feb7c48db2d8b01cd76ebb60cc454f203558f502ef2: Status 404 returned error can't find the container with id 5a37f5ec907f2d04c8275feb7c48db2d8b01cd76ebb60cc454f203558f502ef2 Aug 04 09:08:05.781667 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.781541 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264b1ddb_48ef_4337_87c8_b107b0fce91b.slice/crio-96e5577a7865bd66e9abdda917892fa391a873a8be2cad0f166366aef6f5bd21 WatchSource:0}: Error finding container 96e5577a7865bd66e9abdda917892fa391a873a8be2cad0f166366aef6f5bd21: Status 404 returned error can't find the container with id 96e5577a7865bd66e9abdda917892fa391a873a8be2cad0f166366aef6f5bd21 Aug 04 09:08:05.787383 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.787352 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe78a30_6ac2_4a4f_80b8_166c970b7e62.slice/crio-45e507c5311ca17f3f9fa761c2d4e347b9606bf467dda79a8b037eebc66bc256 WatchSource:0}: Error finding container 45e507c5311ca17f3f9fa761c2d4e347b9606bf467dda79a8b037eebc66bc256: Status 404 returned error can't find the container with id 45e507c5311ca17f3f9fa761c2d4e347b9606bf467dda79a8b037eebc66bc256 Aug 04 09:08:05.788712 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.788408 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe00b519_b049_40dc_a899_3ee10e24a4b4.slice/crio-929277e741cbf62ecb82bf81b9f72f4b20c5e8dcdf04ff2483211cc31e121683 WatchSource:0}: Error finding container 929277e741cbf62ecb82bf81b9f72f4b20c5e8dcdf04ff2483211cc31e121683: Status 404 returned error can't find the container with id 929277e741cbf62ecb82bf81b9f72f4b20c5e8dcdf04ff2483211cc31e121683 Aug 04 09:08:05.789907 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.789513 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f074728_5d3c_4424_a6f7_c9aa4cd6632e.slice/crio-06de9070fd42943543ce1daa1424f13b45cfee81d23a8fab5e58a55e0c3fb15b WatchSource:0}: Error finding container 06de9070fd42943543ce1daa1424f13b45cfee81d23a8fab5e58a55e0c3fb15b: Status 404 returned error can't find the container with id 06de9070fd42943543ce1daa1424f13b45cfee81d23a8fab5e58a55e0c3fb15b Aug 04 09:08:05.790465 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.790381 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8eaff13_86b2_4e77_81a6_e3abc9ebcb4f.slice/crio-d6886f908b6352da5bba5a2c7a972676bd052bbda54907793bb0a90908192bb5 WatchSource:0}: Error finding container d6886f908b6352da5bba5a2c7a972676bd052bbda54907793bb0a90908192bb5: Status 404 returned error can't find the container with id d6886f908b6352da5bba5a2c7a972676bd052bbda54907793bb0a90908192bb5 Aug 04 09:08:05.791553 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.791526 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod661c18ec_bd75_4b19_ad4f_72500024e020.slice/crio-a21cbd31ca082696e676addf77762369055ac7b0669d8b6f1a6183ecc49b4666 WatchSource:0}: Error finding container a21cbd31ca082696e676addf77762369055ac7b0669d8b6f1a6183ecc49b4666: Status 404 returned error can't find the container with id a21cbd31ca082696e676addf77762369055ac7b0669d8b6f1a6183ecc49b4666 Aug 04 09:08:05.792830 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.792795 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00853198_747d_4269_8b81_7081eb8ddf70.slice/crio-4b31db185a4cd38d51e95ce798c393a01c6d1f3d2876ac3c9c6386f23023b888 WatchSource:0}: Error finding container 4b31db185a4cd38d51e95ce798c393a01c6d1f3d2876ac3c9c6386f23023b888: Status 404 returned error can't find the container with id 4b31db185a4cd38d51e95ce798c393a01c6d1f3d2876ac3c9c6386f23023b888 Aug 04 09:08:05.793568 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:05.793506 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7a23ce_b3e7_465c_a7a2_0b713c2b09e7.slice/crio-e5069ee04b0d02969a12cce809f199a8062434d1af25cf18781998fd96edc74d WatchSource:0}: Error finding container e5069ee04b0d02969a12cce809f199a8062434d1af25cf18781998fd96edc74d: Status 404 returned error can't find the container with id e5069ee04b0d02969a12cce809f199a8062434d1af25cf18781998fd96edc74d Aug 04 09:08:05.863787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.863761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:05.863931 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.863894 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:05.863931 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.863918 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:05.863931 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.863928 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:05.864096 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.863979 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:06.863962405 +0000 UTC m=+3.996171921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:05.964752 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.964719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:05.964926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:05.964763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:05.964926 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.964890 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:05.964926 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.964892 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:05.965065 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.964958 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:06.964939555 +0000 UTC m=+4.097149068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:05.965065 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:05.964978 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:08:06.964967754 +0000 UTC m=+4.097177267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:06.331308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.331214 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:03:06 +0000 UTC" deadline="2028-03-28 16:41:37.89841467 +0000 UTC" Aug 04 09:08:06.331308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.331270 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14455h33m31.567163725s" Aug 04 09:08:06.410601 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.410560 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:06.410747 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.410723 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:06.423851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.422649 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8e896688688bfbf603f7c1859e47653" containerID="d794a4f4a1fddfa3a8a0e2d7153168357c3a03cb9539aae6a697314405a2e47c" exitCode=0 Aug 04 09:08:06.423851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.423622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" event={"ID":"c8e896688688bfbf603f7c1859e47653","Type":"ContainerDied","Data":"d794a4f4a1fddfa3a8a0e2d7153168357c3a03cb9539aae6a697314405a2e47c"} Aug 04 09:08:06.428785 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.428756 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lhc2c" event={"ID":"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7","Type":"ContainerStarted","Data":"e5069ee04b0d02969a12cce809f199a8062434d1af25cf18781998fd96edc74d"} Aug 04 09:08:06.433049 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.432948 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"4b31db185a4cd38d51e95ce798c393a01c6d1f3d2876ac3c9c6386f23023b888"} Aug 04 09:08:06.436460 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.436420 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerStarted","Data":"06de9070fd42943543ce1daa1424f13b45cfee81d23a8fab5e58a55e0c3fb15b"} Aug 04 09:08:06.437702 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.437652 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:08:06.440627 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.440563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-74h6b" event={"ID":"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f","Type":"ContainerStarted","Data":"d6886f908b6352da5bba5a2c7a972676bd052bbda54907793bb0a90908192bb5"} Aug 04 09:08:06.452951 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.452923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w6wnq" event={"ID":"fe00b519-b049-40dc-a899-3ee10e24a4b4","Type":"ContainerStarted","Data":"929277e741cbf62ecb82bf81b9f72f4b20c5e8dcdf04ff2483211cc31e121683"} Aug 04 09:08:06.462736 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.462711 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k5fdn" event={"ID":"661c18ec-bd75-4b19-ad4f-72500024e020","Type":"ContainerStarted","Data":"a21cbd31ca082696e676addf77762369055ac7b0669d8b6f1a6183ecc49b4666"} Aug 04 09:08:06.468597 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.468567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" event={"ID":"efe78a30-6ac2-4a4f-80b8-166c970b7e62","Type":"ContainerStarted","Data":"45e507c5311ca17f3f9fa761c2d4e347b9606bf467dda79a8b037eebc66bc256"} Aug 04 09:08:06.476469 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.476441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sf84q" event={"ID":"264b1ddb-48ef-4337-87c8-b107b0fce91b","Type":"ContainerStarted","Data":"96e5577a7865bd66e9abdda917892fa391a873a8be2cad0f166366aef6f5bd21"} Aug 04 09:08:06.480595 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.480530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-22b68" event={"ID":"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f","Type":"ContainerStarted","Data":"5a37f5ec907f2d04c8275feb7c48db2d8b01cd76ebb60cc454f203558f502ef2"} Aug 04 09:08:06.873667 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.873344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:06.873841 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.873721 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:06.873841 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.873746 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:06.873841 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.873761 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:06.873841 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.873819 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:08.873798832 +0000 UTC m=+6.006008346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:06.974546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.974505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:06.974737 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:06.974566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:06.974808 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.974743 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:06.974864 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.974810 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:08.974791541 +0000 UTC m=+6.107001077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:06.975284 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.975239 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:06.975392 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:06.975295 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:08:08.975278935 +0000 UTC m=+6.107488450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:07.332312 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:07.332175 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:03:06 +0000 UTC" deadline="2028-02-01 23:19:49.767033047 +0000 UTC" Aug 04 09:08:07.332312 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:07.332249 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13118h11m42.434790079s" Aug 04 09:08:07.410418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:07.410381 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:07.410604 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:07.410556 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:07.411232 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:07.411075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:07.411232 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:07.411186 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:07.499280 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:07.499224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" event={"ID":"c8e896688688bfbf603f7c1859e47653","Type":"ContainerStarted","Data":"ecb6fbc72a6e0bcc2b379744e6bfc49c32af15b79fbb441acdb6fe56d4f807c6"} Aug 04 09:08:08.410567 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:08.410533 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:08.411069 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.410665 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:08.890119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:08.889363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:08.890119 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.889615 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:08.890119 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.889635 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:08.890119 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.889648 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:08.890119 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.889710 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:12.889689088 +0000 UTC m=+10.021898621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:08.991450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:08.990641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:08.991450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:08.990697 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:08.991450 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.990853 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:08.991450 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.990919 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:12.990899337 +0000 UTC m=+10.123108852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:08.991450 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.991353 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:08.991450 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:08.991412 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:08:12.99139392 +0000 UTC m=+10.123603450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:09.410097 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:09.410065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:09.410097 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:09.410099 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:09.410385 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:09.410227 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:09.410385 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:09.410342 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:10.411061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:10.410528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:10.411061 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:10.410668 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:11.410530 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:11.410495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:11.410713 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:11.410658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:11.410882 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:11.410857 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:11.410937 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:11.410652 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:12.414643 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:12.411514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:12.414643 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:12.411648 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:12.932202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:12.932156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:12.932427 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:12.932382 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:12.932427 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:12.932413 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:12.932427 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:12.932427 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:12.932605 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:12.932500 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:20.932479587 +0000 UTC m=+18.064689100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:13.033131 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:13.032562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:13.033131 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:13.032614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:13.033131 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:13.032740 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:13.033131 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:13.032770 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:13.033131 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:13.032807 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:21.032787672 +0000 UTC m=+18.164997188 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:13.033131 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:13.032837 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:08:21.032818531 +0000 UTC m=+18.165028046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:13.411577 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:13.411505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:13.411737 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:13.411622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:13.411737 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:13.411671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:13.411875 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:13.411775 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:14.410081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:14.410048 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:14.410644 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:14.410167 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:15.410325 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:15.410213 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:15.410795 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:15.410375 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:15.410795 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:15.410383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:15.410795 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:15.410488 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:16.410349 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:16.410314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:16.410770 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:16.410434 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:17.410397 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:17.410358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:17.410836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:17.410368 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:17.410836 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:17.410502 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:17.410836 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:17.410591 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:18.410413 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:18.410382 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:18.410832 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:18.410508 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:19.410287 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:19.410126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:19.410287 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:19.410274 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:19.410821 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:19.410317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:19.410821 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:19.410416 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:20.410769 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:20.410738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:20.411302 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:20.410838 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:20.991915 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:20.991882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:20.992084 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:20.992043 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:20.992084 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:20.992066 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:20.992084 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:20.992076 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:20.992217 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:20.992131 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:36.992113767 +0000 UTC m=+34.124323277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:21.092466 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:21.092428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:21.092656 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:21.092479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:21.092656 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:21.092584 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:21.092656 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:21.092647 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:08:37.09263374 +0000 UTC m=+34.224843253 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:21.092809 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:21.092586 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:21.092809 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:21.092731 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:08:37.09271318 +0000 UTC m=+34.224922701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:21.410586 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:21.410508 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:21.410735 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:21.410524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:21.410735 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:21.410693 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:21.411164 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:21.410797 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:22.410317 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:22.410282 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:22.410484 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:22.410397 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:23.411095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:23.411061 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:23.411564 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:23.411184 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:23.411564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:23.411245 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:23.411564 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:23.411360 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:24.410215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:24.410177 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:24.410406 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:24.410317 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:25.410199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.409898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:25.411092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.409965 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:25.411092 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:25.410316 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:25.411092 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:25.410352 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:25.540058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.539981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k5fdn" event={"ID":"661c18ec-bd75-4b19-ad4f-72500024e020","Type":"ContainerStarted","Data":"6bfe923e547ae9934062b05c9ef7a276b8a5d8402f3906195348af51737702c8"} Aug 04 09:08:25.541839 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.541809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" event={"ID":"efe78a30-6ac2-4a4f-80b8-166c970b7e62","Type":"ContainerStarted","Data":"afb46b7fb6b45e790c239948bc11f230910a4b6a5c53aaf6063834dabc896e4a"} Aug 04 09:08:25.543389 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.543359 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sf84q" event={"ID":"264b1ddb-48ef-4337-87c8-b107b0fce91b","Type":"ContainerStarted","Data":"05772ee688ccb868677c0587c339354d580af2a673829ceac69c841a5f1650b4"} Aug 04 09:08:25.545014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.544981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-22b68" event={"ID":"dbdb2aba-45f2-408b-9c0d-30a4e561dd9f","Type":"ContainerStarted","Data":"3c8a30943763fa39e3657aaa866ea3c1fadbb6169d4f494a72fb0e0c43dc1893"} Aug 04 09:08:25.546445 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.546417 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" event={"ID":"2c069c649a5d879b02b53f674cdd665d","Type":"ContainerStarted","Data":"850c611f43999a87c145660bc63b490cd176cdb2c1fd58b44133bfadf05cc72b"} Aug 04 09:08:25.547999 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.547973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lhc2c" event={"ID":"7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7","Type":"ContainerStarted","Data":"2b7521691c016ee1c4d9be891c5edea3bc838dd7045388a7a65018ed4a6b9b10"} Aug 04 09:08:25.553276 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.553238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"55b90d450f63630986f6ae1c1d0ac7645e24233e87a4ec89ae80a4a651682720"} Aug 04 09:08:25.553384 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.553327 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"b3caf5be7027d21b1c5d2dde363e4300bcc0039a14b7cd2f993650214d4f2142"} Aug 04 09:08:25.553384 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.553375 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"66fe14f93efcd2fd4c20cf7cbad5f21fbd03bcb870017e41665ccc2bff914beb"} Aug 04 09:08:25.553492 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.553423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"8247992fc9da592b53d8b885b859382134782fbbe35f29441e16a2ccf4abf903"} Aug 04 09:08:25.553492 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.553472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"486be6a42b2ea25e29a884b98e0faa8984f5ebaa1cb5d1b34876a0c3e5674b22"} Aug 04 09:08:25.553592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.553518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"e478de1e6b9c47307869fccdeec628ebf9d223d9dcb5d06e1f083ff5bd226e88"} Aug 04 09:08:25.554870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.554844 2562 generic.go:356] "Generic (PLEG): container finished" podID="4f074728-5d3c-4424-a6f7-c9aa4cd6632e" containerID="f9d99af8f994a6c209a712893b0ff2c5d8c70f5519f06cea24f8c8f10f86101e" exitCode=0 Aug 04 09:08:25.554967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.554920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerDied","Data":"f9d99af8f994a6c209a712893b0ff2c5d8c70f5519f06cea24f8c8f10f86101e"} Aug 04 09:08:25.559277 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.557800 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k5fdn" podStartSLOduration=2.578549367 podStartE2EDuration="21.557775422s" totalImagesPullingTime="18.979226055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:25.557049099 +0000 UTC m=+22.689258609" watchObservedRunningTime="2026-08-04 09:08:25.557775422 +0000 UTC m=+22.689984957" Aug 04 09:08:25.559277 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.558051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-74h6b" event={"ID":"e8eaff13-86b2-4e77-81a6-e3abc9ebcb4f","Type":"ContainerStarted","Data":"29234a971b58a074e2c5251cdf1707fddf49fdb3d706cd121fc2d9a8b1d5334e"} Aug 04 09:08:25.559277 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.558307 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-20.ec2.internal" podStartSLOduration=20.55829872 podStartE2EDuration="20.55829872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:07.519021176 +0000 UTC m=+4.651230712" watchObservedRunningTime="2026-08-04 09:08:25.55829872 +0000 UTC m=+22.690508252" Aug 04 09:08:25.620558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.620515 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sf84q" podStartSLOduration=2.657962804 podStartE2EDuration="21.620502224s" totalImagesPullingTime="18.96253942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:25.620025599 +0000 UTC m=+22.752235146" watchObservedRunningTime="2026-08-04 09:08:25.620502224 +0000 UTC m=+22.752711755" Aug 04 09:08:25.638634 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.638597 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lhc2c" podStartSLOduration=2.746304519 podStartE2EDuration="21.638581252s" totalImagesPullingTime="18.892276733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:25.638539646 +0000 UTC m=+22.770749178" watchObservedRunningTime="2026-08-04 09:08:25.638581252 +0000 UTC m=+22.770790783" Aug 04 09:08:25.653982 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.653948 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-22b68" podStartSLOduration=2.687638289 podStartE2EDuration="21.653936959s" totalImagesPullingTime="18.96629867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:25.653864969 +0000 UTC m=+22.786074505" watchObservedRunningTime="2026-08-04 09:08:25.653936959 +0000 UTC m=+22.786146492" Aug 04 09:08:25.670033 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.669989 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-20.ec2.internal" podStartSLOduration=20.669977414 podStartE2EDuration="20.669977414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:25.669949664 +0000 UTC m=+22.802159219" watchObservedRunningTime="2026-08-04 09:08:25.669977414 +0000 UTC m=+22.802186947" Aug 04 09:08:25.690874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.690834 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-74h6b" podStartSLOduration=2.710570034 podStartE2EDuration="21.690823038s" totalImagesPullingTime="18.980253004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:25.690723132 +0000 UTC m=+22.822932657" watchObservedRunningTime="2026-08-04 09:08:25.690823038 +0000 UTC m=+22.823032569" Aug 04 09:08:25.717155 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.717127 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:25.822045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.821986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:25.884264 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:25.884224 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:26.410226 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:26.410197 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:26.410755 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:26.410336 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:26.560891 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:26.560863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w6wnq" event={"ID":"fe00b519-b049-40dc-a899-3ee10e24a4b4","Type":"ContainerStarted","Data":"1cd1cebc402fc9bb505f03d293dd8bba2f03fc7d1779ab39e26c18831c675f77"} Aug 04 09:08:26.576431 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:26.576408 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 09:08:26.576849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:26.576809 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w6wnq" podStartSLOduration=3.621050556 podStartE2EDuration="22.576797232s" totalImagesPullingTime="18.955746676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:26.576013628 +0000 UTC m=+23.708223171" watchObservedRunningTime="2026-08-04 09:08:26.576797232 +0000 UTC m=+23.709006764" Aug 04 09:08:26.696429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:26.696359 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:26.796976 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:26.796949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:27.110920 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.110842 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:27.111479 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.111458 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:27.165231 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.165201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:27.218066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.218037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:27.313102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.313008 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T09:08:26.576426601Z","UUID":"11349aaa-272f-4983-bc9a-f19bc3460b65","Handler":null,"Name":"","Endpoint":""} Aug 04 09:08:27.314826 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.314799 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 09:08:27.314947 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.314837 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 09:08:27.409987 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.409960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:27.410096 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:27.410073 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:27.410151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.410102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:27.410192 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:27.410179 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:27.430533 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.430499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:27.512483 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.512449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:27.567571 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.567533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"fd0ea07afe7ca06672368e4f54b975bcd82318bc1bc8b4f741619d2742971af4"} Aug 04 09:08:27.570227 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.570194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" event={"ID":"efe78a30-6ac2-4a4f-80b8-166c970b7e62","Type":"ContainerStarted","Data":"93533ae6fae63797ae9d925314264c567905f43883118cf083dcac53ee81504b"} Aug 04 09:08:27.570671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.570615 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:27.571188 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.571168 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-22b68" Aug 04 09:08:27.640982 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.640903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:27.742640 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:27.742607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:28.359368 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:28.359297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:28.410163 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:28.410134 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:28.410401 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:28.410247 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:28.574443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:28.574394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" event={"ID":"efe78a30-6ac2-4a4f-80b8-166c970b7e62","Type":"ContainerStarted","Data":"994fefbf1bf4872ad474a384d3ac4b3b2af721177a49df831219d210018a147f"} Aug 04 09:08:28.606085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:28.606042 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbhk7" podStartSLOduration=2.7972068070000002 podStartE2EDuration="24.606027002s" totalImagesPullingTime="21.808820195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:28.604879399 +0000 UTC m=+25.737088929" watchObservedRunningTime="2026-08-04 09:08:28.606027002 +0000 UTC m=+25.738236535" Aug 04 09:08:29.410532 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.410324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:29.410706 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:29.410621 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:29.410827 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.410403 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:29.410955 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:29.410933 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:29.585117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.583063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" event={"ID":"00853198-747d-4269-8b81-7081eb8ddf70","Type":"ContainerStarted","Data":"f136338010638d5ece5c12d2761aae972d0e280c25cc445efb308a2150588013"} Aug 04 09:08:29.585117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.583387 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:29.585117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.583464 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:29.585117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.583514 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:29.603864 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.603841 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:29.604225 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.604145 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:08:29.664705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:29.664608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" podStartSLOduration=6.641682285 podStartE2EDuration="25.664588137s" totalImagesPullingTime="19.022905852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:29.62571046 +0000 UTC m=+26.757919982" watchObservedRunningTime="2026-08-04 09:08:29.664588137 +0000 UTC m=+26.796797670" Aug 04 09:08:30.410325 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:30.410297 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:30.410486 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:30.410388 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:31.410439 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:31.410404 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:31.411100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:31.410568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:31.411100 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:31.410574 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:31.411100 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:31.410730 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:32.155842 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:32.155592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:32.410646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:32.410566 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:32.411085 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:32.410669 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:32.589562 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:32.589520 2562 generic.go:356] "Generic (PLEG): container finished" podID="4f074728-5d3c-4424-a6f7-c9aa4cd6632e" containerID="09ca59c9a7036df975c7970b85fc7d9277d646ba0166d28662da3c9cc5ba3dbb" exitCode=0 Aug 04 09:08:32.589713 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:32.589576 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerDied","Data":"09ca59c9a7036df975c7970b85fc7d9277d646ba0166d28662da3c9cc5ba3dbb"} Aug 04 09:08:32.903503 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:32.903471 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:33.042122 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.042088 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:33.145107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.145069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:33.241914 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.241817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:33.342767 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.342745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:33.410975 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.410942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:33.411392 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.411016 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:33.411392 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:33.411038 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:33.411392 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:33.411112 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:33.443609 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.443588 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:33.593381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.593318 2562 generic.go:356] "Generic (PLEG): container finished" podID="4f074728-5d3c-4424-a6f7-c9aa4cd6632e" containerID="1c0efc2213278c55fe95176bc0d19ad675add8657c510439809038246bebd1b1" exitCode=0 Aug 04 09:08:33.593381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.593368 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerDied","Data":"1c0efc2213278c55fe95176bc0d19ad675add8657c510439809038246bebd1b1"} Aug 04 09:08:33.595510 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:33.595488 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:34.401476 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:34.401452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:34.410023 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:34.410000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:34.410111 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:34.410094 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:34.597946 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:34.597866 2562 generic.go:356] "Generic (PLEG): container finished" podID="4f074728-5d3c-4424-a6f7-c9aa4cd6632e" containerID="96ff815ee3525dabbc3b7d0cc7415cbfe6f6859f516156b1ed756207cded79c3" exitCode=0 Aug 04 09:08:34.598288 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:34.597937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerDied","Data":"96ff815ee3525dabbc3b7d0cc7415cbfe6f6859f516156b1ed756207cded79c3"} Aug 04 09:08:34.790611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:34.790582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:34.984811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:34.984780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:35.395342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:35.395312 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:35.410539 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:35.410511 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:35.410539 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:35.410533 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:35.410673 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:35.410616 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:35.410785 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:35.410763 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:36.183119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:36.183071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:36.388784 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:36.388739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:36.410844 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:36.410815 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:36.410974 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:36.410935 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:36.774990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:36.774950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:36.830660 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:36.830625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:36.997216 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:36.997179 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:37.013270 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.013230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:37.013434 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.013416 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:08:37.013504 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.013440 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:08:37.013504 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.013454 2562 projected.go:196] Error preparing data for projected volume kube-api-access-ctc8s for pod openshift-network-diagnostics/network-check-target-zcjs4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:37.013585 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.013516 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s podName:057e8a4f-5918-497d-9232-1ce627576f54 nodeName:}" failed. No retries permitted until 2026-08-04 09:09:09.013496475 +0000 UTC m=+66.145706004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ctc8s" (UniqueName: "kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s") pod "network-check-target-zcjs4" (UID: "057e8a4f-5918-497d-9232-1ce627576f54") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:08:37.114529 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.114449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:37.114529 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.114487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:37.114742 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.114609 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:37.114742 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.114665 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret podName:bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc nodeName:}" failed. No retries permitted until 2026-08-04 09:09:09.114651743 +0000 UTC m=+66.246861255 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret") pod "global-pull-secret-syncer-cnb48" (UID: "bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:08:37.114742 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.114609 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:37.114742 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.114705 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs podName:9f2c9e63-ede8-4b44-9f40-6e8370313c84 nodeName:}" failed. No retries permitted until 2026-08-04 09:09:09.114696266 +0000 UTC m=+66.246905780 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs") pod "network-metrics-daemon-t72c2" (UID: "9f2c9e63-ede8-4b44-9f40-6e8370313c84") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:08:37.410497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.410463 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:37.410955 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.410607 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:37.410955 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.410661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:37.410955 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:37.410772 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:37.582551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.582521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:37.684564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.684486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:37.927599 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:37.927570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:38.000480 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:38.000401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:38.410440 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:38.410412 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:38.410600 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:38.410515 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:38.841570 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:38.841482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:38.941091 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:38.941058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.041075 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.041033 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.183416 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.183385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.298559 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.298530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.399429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.399406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.410494 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.410473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:39.410597 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:39.410578 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:39.410642 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.410629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:39.410921 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:39.410711 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:39.523301 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.523272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.611323 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.611292 2562 generic.go:356] "Generic (PLEG): container finished" podID="4f074728-5d3c-4424-a6f7-c9aa4cd6632e" containerID="b7bb09745791e75fde13a921de7aa3dabeca77c8d5e3ba0fd9ac10496a374267" exitCode=0 Aug 04 09:08:39.611459 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.611346 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerDied","Data":"b7bb09745791e75fde13a921de7aa3dabeca77c8d5e3ba0fd9ac10496a374267"} Aug 04 09:08:39.625645 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.625618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.691746 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.691720 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:39.763441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:39.763413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:40.189050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.189014 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:40.290667 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.290631 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:40.390342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.390306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:40.410355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.410340 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:40.410444 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:40.410428 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:40.559108 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.559042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:40.616466 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.616432 2562 generic.go:356] "Generic (PLEG): container finished" podID="4f074728-5d3c-4424-a6f7-c9aa4cd6632e" containerID="a998ae55141cb1ae45291b1a1b8e22c1814d49ed519846a6f0f1de3aea607389" exitCode=0 Aug 04 09:08:40.616591 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.616484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerDied","Data":"a998ae55141cb1ae45291b1a1b8e22c1814d49ed519846a6f0f1de3aea607389"} Aug 04 09:08:40.786387 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.786356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:40.954399 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:40.954371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:41.055036 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.055008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:41.396725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.396699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:41.410786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.410764 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:41.410888 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:41.410869 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:41.410950 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.410934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:41.411052 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:41.411035 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:41.589522 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.589495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:41.622160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.622128 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" event={"ID":"4f074728-5d3c-4424-a6f7-c9aa4cd6632e","Type":"ContainerStarted","Data":"03a0157a90c95c7d0c3908fa96c711ffeaf93b031e69eac5724639e6ba6903ee"} Aug 04 09:08:41.648443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:41.648366 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b4vcj" podStartSLOduration=6.819294025 podStartE2EDuration="37.648352179s" totalImagesPullingTime="30.829058154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:41.647227439 +0000 UTC m=+38.779436983" watchObservedRunningTime="2026-08-04 09:08:41.648352179 +0000 UTC m=+38.780561711" Aug 04 09:08:42.022602 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:42.022526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:42.193452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:42.193406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:42.410174 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:42.410153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:42.410316 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:42.410239 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:42.586970 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:42.586939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:43.201095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.201056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:43.410524 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.410495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:43.410680 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:43.410585 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:43.410725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.410675 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:43.410808 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:43.410791 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:43.590604 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.590508 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:43.672182 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.672155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:43.743420 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.743367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:43.806685 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.806659 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:43.988459 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:43.988429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:44.410172 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:44.410152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:44.410543 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:44.410243 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:45.410407 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:45.410377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:45.410407 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:45.410397 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:45.410801 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:45.410476 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:45.410801 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:45.410622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:46.409904 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:46.409873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:46.410062 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:46.409969 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:47.410164 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:47.410132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:47.410557 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:47.410269 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:47.410557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:47.410313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:47.410557 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:47.410399 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:47.420464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:47.420435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:47.634272 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:47.634225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:47.831921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:47.831826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:47.997148 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:47.997117 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:48.134936 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:48.134865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:48.410569 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:48.410544 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:48.410898 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:48.410633 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:48.595870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:48.595842 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:48.686710 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:48.686642 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:49.140376 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:49.140129 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cnb48"] Aug 04 09:08:49.140801 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:49.140425 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:49.140930 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:49.140905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:49.142574 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:49.142549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zcjs4"] Aug 04 09:08:49.142677 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:49.142649 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:49.142801 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:49.142757 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:49.143723 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:49.143701 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t72c2"] Aug 04 09:08:49.143828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:49.143797 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:49.143922 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:49.143901 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:50.410726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:50.410698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:50.411234 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:50.410802 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:51.410088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:51.410051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:51.410295 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:51.410182 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:51.410295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:51.410229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:51.410434 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:51.410362 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:51.720151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:51.720078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:51.825922 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:51.825894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:52.409828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:52.409792 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:52.409996 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:52.409908 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t72c2" podUID="9f2c9e63-ede8-4b44-9f40-6e8370313c84" Aug 04 09:08:53.412487 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.412465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:53.412943 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.412542 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:53.412943 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:53.412573 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcjs4" podUID="057e8a4f-5918-497d-9232-1ce627576f54" Aug 04 09:08:53.412943 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:08:53.412592 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cnb48" podUID="bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc" Aug 04 09:08:53.519131 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.519052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:53.610641 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.610614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:53.715126 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.715097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:53.748709 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.748683 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-20.ec2.internal" event="NodeReady" Aug 04 09:08:53.748872 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.748815 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 09:08:53.822962 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.822895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:53.834976 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.834945 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4tpsh"] Aug 04 09:08:53.848851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.848827 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cv7j8"] Aug 04 09:08:53.848851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.848838 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:53.851511 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.851473 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 09:08:53.851644 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.851625 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 09:08:53.851970 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.851955 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 09:08:53.853699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.853685 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lkd2q\"" Aug 04 09:08:53.855816 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.855799 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:53.861374 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.861355 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4tpsh"] Aug 04 09:08:53.861662 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.861631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 09:08:53.861772 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.861711 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 09:08:53.866242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.866216 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-56lhf\"" Aug 04 09:08:53.866242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.866227 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 09:08:53.866400 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.866353 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 09:08:53.875833 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.875815 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cv7j8"] Aug 04 09:08:53.926188 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.926165 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vg8zv"] Aug 04 09:08:53.936936 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.936917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:53.940149 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7bqc\" (UniqueName: \"kubernetes.io/projected/be8cc727-6540-4150-93cf-9e35055d8f4e-kube-api-access-w7bqc\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:53.940266 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be8cc727-6540-4150-93cf-9e35055d8f4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:53.940266 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be8cc727-6540-4150-93cf-9e35055d8f4e-crio-socket\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:53.940376 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqddb\" (UniqueName: \"kubernetes.io/projected/e4ae14d4-e5c9-43d0-89ef-22edc19a7255-kube-api-access-xqddb\") pod \"ingress-canary-4tpsh\" (UID: \"e4ae14d4-e5c9-43d0-89ef-22edc19a7255\") " pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:53.940376 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be8cc727-6540-4150-93cf-9e35055d8f4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:53.940376 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4ae14d4-e5c9-43d0-89ef-22edc19a7255-cert\") pod \"ingress-canary-4tpsh\" (UID: \"e4ae14d4-e5c9-43d0-89ef-22edc19a7255\") " pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:53.940472 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.940413 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be8cc727-6540-4150-93cf-9e35055d8f4e-data-volume\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:53.941840 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.941823 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xnmk\"" Aug 04 09:08:53.942019 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.942001 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 09:08:53.942128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.942113 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 09:08:53.953087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.953068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vg8zv"] Aug 04 09:08:53.960783 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:53.960755 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:54.041401 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd275c7d-a708-4177-ad02-170170c21010-config-volume\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.041493 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041417 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be8cc727-6540-4150-93cf-9e35055d8f4e-data-volume\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041493 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7bqc\" (UniqueName: \"kubernetes.io/projected/be8cc727-6540-4150-93cf-9e35055d8f4e-kube-api-access-w7bqc\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041493 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd275c7d-a708-4177-ad02-170170c21010-tmp-dir\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.041592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be8cc727-6540-4150-93cf-9e35055d8f4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd275c7d-a708-4177-ad02-170170c21010-metrics-tls\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.041680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb5nc\" (UniqueName: \"kubernetes.io/projected/fd275c7d-a708-4177-ad02-170170c21010-kube-api-access-sb5nc\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.041680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be8cc727-6540-4150-93cf-9e35055d8f4e-crio-socket\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqddb\" (UniqueName: \"kubernetes.io/projected/e4ae14d4-e5c9-43d0-89ef-22edc19a7255-kube-api-access-xqddb\") pod \"ingress-canary-4tpsh\" (UID: \"e4ae14d4-e5c9-43d0-89ef-22edc19a7255\") " pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:54.041797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be8cc727-6540-4150-93cf-9e35055d8f4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041726 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be8cc727-6540-4150-93cf-9e35055d8f4e-data-volume\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041772 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be8cc727-6540-4150-93cf-9e35055d8f4e-crio-socket\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.041907 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.041864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4ae14d4-e5c9-43d0-89ef-22edc19a7255-cert\") pod \"ingress-canary-4tpsh\" (UID: \"e4ae14d4-e5c9-43d0-89ef-22edc19a7255\") " pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:54.042169 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.042151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be8cc727-6540-4150-93cf-9e35055d8f4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.045408 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.045391 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be8cc727-6540-4150-93cf-9e35055d8f4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.051319 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.051300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4ae14d4-e5c9-43d0-89ef-22edc19a7255-cert\") pod \"ingress-canary-4tpsh\" (UID: \"e4ae14d4-e5c9-43d0-89ef-22edc19a7255\") " pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:54.051404 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.051318 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7bqc\" (UniqueName: \"kubernetes.io/projected/be8cc727-6540-4150-93cf-9e35055d8f4e-kube-api-access-w7bqc\") pod \"insights-runtime-extractor-cv7j8\" (UID: \"be8cc727-6540-4150-93cf-9e35055d8f4e\") " pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.051449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.051417 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqddb\" (UniqueName: \"kubernetes.io/projected/e4ae14d4-e5c9-43d0-89ef-22edc19a7255-kube-api-access-xqddb\") pod \"ingress-canary-4tpsh\" (UID: \"e4ae14d4-e5c9-43d0-89ef-22edc19a7255\") " pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:54.073268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.073201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:54.142765 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.142741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd275c7d-a708-4177-ad02-170170c21010-tmp-dir\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.142867 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.142780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd275c7d-a708-4177-ad02-170170c21010-metrics-tls\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.142867 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.142812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb5nc\" (UniqueName: \"kubernetes.io/projected/fd275c7d-a708-4177-ad02-170170c21010-kube-api-access-sb5nc\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.142867 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.142840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd275c7d-a708-4177-ad02-170170c21010-config-volume\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.143123 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.143104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd275c7d-a708-4177-ad02-170170c21010-tmp-dir\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.143320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.143303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fd275c7d-a708-4177-ad02-170170c21010-config-volume\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.144872 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.144858 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd275c7d-a708-4177-ad02-170170c21010-metrics-tls\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.154101 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.154077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb5nc\" (UniqueName: \"kubernetes.io/projected/fd275c7d-a708-4177-ad02-170170c21010-kube-api-access-sb5nc\") pod \"dns-default-vg8zv\" (UID: \"fd275c7d-a708-4177-ad02-170170c21010\") " pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.158014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.157998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4tpsh" Aug 04 09:08:54.165168 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.165152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cv7j8" Aug 04 09:08:54.231278 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.231236 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:54.245517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.245496 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:54.370946 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.370760 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4tpsh"] Aug 04 09:08:54.374948 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:54.374868 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ae14d4_e5c9_43d0_89ef_22edc19a7255.slice/crio-49367346c4b38a8611012bd07a82382698383350603a54d1ba1f1603c3f436bf WatchSource:0}: Error finding container 49367346c4b38a8611012bd07a82382698383350603a54d1ba1f1603c3f436bf: Status 404 returned error can't find the container with id 49367346c4b38a8611012bd07a82382698383350603a54d1ba1f1603c3f436bf Aug 04 09:08:54.375316 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.375292 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cv7j8"] Aug 04 09:08:54.378203 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:54.378183 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8cc727_6540_4150_93cf_9e35055d8f4e.slice/crio-87aaa0d92097df2852b8fa7291eb26cad9ac39ea38ea3d884307e5b124fffdb9 WatchSource:0}: Error finding container 87aaa0d92097df2852b8fa7291eb26cad9ac39ea38ea3d884307e5b124fffdb9: Status 404 returned error can't find the container with id 87aaa0d92097df2852b8fa7291eb26cad9ac39ea38ea3d884307e5b124fffdb9 Aug 04 09:08:54.408209 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.408188 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vg8zv"] Aug 04 09:08:54.410376 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.410358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:08:54.413138 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:08:54.413115 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd275c7d_a708_4177_ad02_170170c21010.slice/crio-28e75ac85620462346740920f36dfdaf088b323ae1295edb9c3da68cf37c71f5 WatchSource:0}: Error finding container 28e75ac85620462346740920f36dfdaf088b323ae1295edb9c3da68cf37c71f5: Status 404 returned error can't find the container with id 28e75ac85620462346740920f36dfdaf088b323ae1295edb9c3da68cf37c71f5 Aug 04 09:08:54.413792 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.413771 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djwk2\"" Aug 04 09:08:54.413792 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.413775 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 09:08:54.651790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.651753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vg8zv" event={"ID":"fd275c7d-a708-4177-ad02-170170c21010","Type":"ContainerStarted","Data":"28e75ac85620462346740920f36dfdaf088b323ae1295edb9c3da68cf37c71f5"} Aug 04 09:08:54.653197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.653167 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cv7j8" event={"ID":"be8cc727-6540-4150-93cf-9e35055d8f4e","Type":"ContainerStarted","Data":"086d2629abefc977fca5ae3e1bf1a39f59452d9a92c41841fd5713e24374621d"} Aug 04 09:08:54.653343 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.653220 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cv7j8" event={"ID":"be8cc727-6540-4150-93cf-9e35055d8f4e","Type":"ContainerStarted","Data":"87aaa0d92097df2852b8fa7291eb26cad9ac39ea38ea3d884307e5b124fffdb9"} Aug 04 09:08:54.654212 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:54.654182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4tpsh" event={"ID":"e4ae14d4-e5c9-43d0-89ef-22edc19a7255","Type":"ContainerStarted","Data":"49367346c4b38a8611012bd07a82382698383350603a54d1ba1f1603c3f436bf"} Aug 04 09:08:55.410518 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.410485 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:08:55.410679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.410501 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:08:55.413387 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.413365 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 09:08:55.414531 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.414247 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 09:08:55.414531 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.414284 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 09:08:55.414531 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.414247 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jr9km\"" Aug 04 09:08:55.659803 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.659765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cv7j8" event={"ID":"be8cc727-6540-4150-93cf-9e35055d8f4e","Type":"ContainerStarted","Data":"ecebb73c4213c5e5085cd0aa8bc88c1ff67864996856b48f9426969eb6a8f22d"} Aug 04 09:08:55.985185 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:55.984993 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:56.105340 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:56.105311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:56.773118 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:56.773040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:56.844159 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:56.844122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:57.493731 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.493621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:57.592498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.592467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:57.668261 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.668229 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cv7j8" event={"ID":"be8cc727-6540-4150-93cf-9e35055d8f4e","Type":"ContainerStarted","Data":"aa81bfd81f6a96996c39727a0ebfc61607910a6f124eef803abaa1ffaeb66bed"} Aug 04 09:08:57.669730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.669702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4tpsh" event={"ID":"e4ae14d4-e5c9-43d0-89ef-22edc19a7255","Type":"ContainerStarted","Data":"f18db7b29cc7348667a2debd83c8d27becf2e52d4426b6367e5df68532c91418"} Aug 04 09:08:57.671338 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.671315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vg8zv" event={"ID":"fd275c7d-a708-4177-ad02-170170c21010","Type":"ContainerStarted","Data":"d71f29e44444f34f2513b1c9d81ca5a4f04ffd147e7ba104597103f75ce86be1"} Aug 04 09:08:57.671442 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.671361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vg8zv" event={"ID":"fd275c7d-a708-4177-ad02-170170c21010","Type":"ContainerStarted","Data":"b8b4d8d2cba011ca2e43ced0b49845636faaaef915abce5c8dd47aa595aca334"} Aug 04 09:08:57.671442 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.671401 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vg8zv" Aug 04 09:08:57.687125 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.687090 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cv7j8" podStartSLOduration=1.863826663 podStartE2EDuration="4.687078121s" totalImagesPullingTime="2.823251458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:57.686202477 +0000 UTC m=+54.818412008" watchObservedRunningTime="2026-08-04 09:08:57.687078121 +0000 UTC m=+54.819287647" Aug 04 09:08:57.702455 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.702023 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4tpsh" podStartSLOduration=1.683716543 podStartE2EDuration="4.702008369s" totalImagesPullingTime="3.018291826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:57.701413486 +0000 UTC m=+54.833623017" watchObservedRunningTime="2026-08-04 09:08:57.702008369 +0000 UTC m=+54.834217885" Aug 04 09:08:57.719941 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:57.719893 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vg8zv" podStartSLOduration=1.7616230480000001 podStartE2EDuration="4.719874616s" totalImagesPullingTime="2.958251568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:08:57.718844628 +0000 UTC m=+54.851054181" watchObservedRunningTime="2026-08-04 09:08:57.719874616 +0000 UTC m=+54.852084149" Aug 04 09:08:59.308707 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:59.308652 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:08:59.410158 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:08:59.410129 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:01.373805 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:01.373761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:01.448239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:01.448206 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:01.604298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:01.604269 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-857rg" Aug 04 09:09:01.795958 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:01.795903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:01.897367 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:01.897327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:06.504647 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:06.504581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:06.604133 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:06.604084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:07.677418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:07.677388 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vg8zv" Aug 04 09:09:07.817813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:07.817781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:07.918592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:07.918558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:09.045425 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.045381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:09:09.048162 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.048140 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 09:09:09.058098 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.058081 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 09:09:09.069987 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.069964 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctc8s\" (UniqueName: \"kubernetes.io/projected/057e8a4f-5918-497d-9232-1ce627576f54-kube-api-access-ctc8s\") pod \"network-check-target-zcjs4\" (UID: \"057e8a4f-5918-497d-9232-1ce627576f54\") " pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:09:09.146201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.146167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:09:09.146336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.146209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:09:09.148492 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.148475 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 09:09:09.149135 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.149118 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 09:09:09.158914 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.158896 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f2c9e63-ede8-4b44-9f40-6e8370313c84-metrics-certs\") pod \"network-metrics-daemon-t72c2\" (UID: \"9f2c9e63-ede8-4b44-9f40-6e8370313c84\") " pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:09:09.158990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.158959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc-original-pull-secret\") pod \"global-pull-secret-syncer-cnb48\" (UID: \"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc\") " pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:09:09.222383 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.222351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cnb48" Aug 04 09:09:09.249107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.249085 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jr9km\"" Aug 04 09:09:09.256763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.256720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:09:09.379525 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.379496 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cnb48"] Aug 04 09:09:09.382408 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:09.382381 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbbd5193_de5e_4b69_bb4c_94ce6f8c0ecc.slice/crio-86b287590f260aaef19517fc232f59b702c1e922faa547fc6dcadc3f0bed49ad WatchSource:0}: Error finding container 86b287590f260aaef19517fc232f59b702c1e922faa547fc6dcadc3f0bed49ad: Status 404 returned error can't find the container with id 86b287590f260aaef19517fc232f59b702c1e922faa547fc6dcadc3f0bed49ad Aug 04 09:09:09.393671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.393599 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zcjs4"] Aug 04 09:09:09.397169 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:09.397148 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057e8a4f_5918_497d_9232_1ce627576f54.slice/crio-be73343a9468262ba38d9f67094dcb587110ef75e4a7704a0a1d2b826d211742 WatchSource:0}: Error finding container be73343a9468262ba38d9f67094dcb587110ef75e4a7704a0a1d2b826d211742: Status 404 returned error can't find the container with id be73343a9468262ba38d9f67094dcb587110ef75e4a7704a0a1d2b826d211742 Aug 04 09:09:09.428087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.428063 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djwk2\"" Aug 04 09:09:09.436313 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.436292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t72c2" Aug 04 09:09:09.585973 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.585900 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t72c2"] Aug 04 09:09:09.588873 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:09.588844 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2c9e63_ede8_4b44_9f40_6e8370313c84.slice/crio-ea4fe5f4cd1263f8b9316afa5515dc55226c7cb6f8535a8f16e4820332f143fd WatchSource:0}: Error finding container ea4fe5f4cd1263f8b9316afa5515dc55226c7cb6f8535a8f16e4820332f143fd: Status 404 returned error can't find the container with id ea4fe5f4cd1263f8b9316afa5515dc55226c7cb6f8535a8f16e4820332f143fd Aug 04 09:09:09.711028 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.710989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t72c2" event={"ID":"9f2c9e63-ede8-4b44-9f40-6e8370313c84","Type":"ContainerStarted","Data":"ea4fe5f4cd1263f8b9316afa5515dc55226c7cb6f8535a8f16e4820332f143fd"} Aug 04 09:09:09.712016 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.711992 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zcjs4" event={"ID":"057e8a4f-5918-497d-9232-1ce627576f54","Type":"ContainerStarted","Data":"be73343a9468262ba38d9f67094dcb587110ef75e4a7704a0a1d2b826d211742"} Aug 04 09:09:09.713076 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.713051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cnb48" event={"ID":"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc","Type":"ContainerStarted","Data":"86b287590f260aaef19517fc232f59b702c1e922faa547fc6dcadc3f0bed49ad"} Aug 04 09:09:09.738327 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.738302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:09.843533 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.843443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:09.938375 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:09.938343 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:10.373590 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:10.373556 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:10.483083 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:10.483016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:10.587953 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:10.587883 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:10.954115 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:10.954080 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:11.573539 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:11.573501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:11.724374 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:11.724335 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t72c2" event={"ID":"9f2c9e63-ede8-4b44-9f40-6e8370313c84","Type":"ContainerStarted","Data":"ef668ddb1b5c7e54c9055d82263e384fc555a92878bdd07c572f89ca8d2f365f"} Aug 04 09:09:11.724552 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:11.724403 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t72c2" event={"ID":"9f2c9e63-ede8-4b44-9f40-6e8370313c84","Type":"ContainerStarted","Data":"ec2d58a03cf7a894b09ac8af90a51f1f2528872bff1cb80bc9c0038a328aed5e"} Aug 04 09:09:11.757972 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:11.757915 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t72c2" podStartSLOduration=66.458038581 podStartE2EDuration="1m7.757899644s" totalImagesPullingTime="1.299861063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:11.756180417 +0000 UTC m=+68.888389948" watchObservedRunningTime="2026-08-04 09:09:11.757899644 +0000 UTC m=+68.890109215" Aug 04 09:09:11.895469 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:11.895430 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:12.005795 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.005751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:12.206860 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.206772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:12.438667 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.438627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:12.617182 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.617028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:12.636903 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.636855 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf"] Aug 04 09:09:12.639878 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.639853 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:12.643474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.643444 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-slnqd\"" Aug 04 09:09:12.643813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.643793 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 04 09:09:12.663337 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.663306 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf"] Aug 04 09:09:12.673105 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.673080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-sr8qf\" (UID: \"b0cd79c3-d796-4f80-875c-9e08fe208f7a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:12.773656 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.773609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-sr8qf\" (UID: \"b0cd79c3-d796-4f80-875c-9e08fe208f7a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:12.773834 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:12.773743 2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Aug 04 09:09:12.773834 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:12.773808 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates podName:b0cd79c3-d796-4f80-875c-9e08fe208f7a nodeName:}" failed. No retries permitted until 2026-08-04 09:09:13.27378935 +0000 UTC m=+70.405998880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-sr8qf" (UID: "b0cd79c3-d796-4f80-875c-9e08fe208f7a") : secret "prometheus-operator-admission-webhook-tls" not found Aug 04 09:09:12.785980 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.785948 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db4778974-llvgp"] Aug 04 09:09:12.788133 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.788107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.794337 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.794305 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 04 09:09:12.795158 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.794983 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gxlll\"" Aug 04 09:09:12.795413 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.795392 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 04 09:09:12.795551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.795501 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 09:09:12.795611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.795602 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 09:09:12.795760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.795742 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 04 09:09:12.795841 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.795746 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 04 09:09:12.796401 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.796234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 04 09:09:12.806618 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.806592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db4778974-llvgp"] Aug 04 09:09:12.874927 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.874843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-przlx\" (UniqueName: \"kubernetes.io/projected/bba28cab-660e-43f1-abf1-471e2cc17889-kube-api-access-przlx\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.874927 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.874903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-service-ca\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.875169 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.874935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-serving-cert\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.875169 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.874963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-oauth-config\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.875169 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.875026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-console-config\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.875169 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.875047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-oauth-serving-cert\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.976350 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.976308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-console-config\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.976544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.976357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-oauth-serving-cert\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.976544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.976393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-przlx\" (UniqueName: \"kubernetes.io/projected/bba28cab-660e-43f1-abf1-471e2cc17889-kube-api-access-przlx\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.976544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.976525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-service-ca\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.976700 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.976607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-serving-cert\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.976700 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.976636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-oauth-config\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.977198 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.977172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-console-config\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.977327 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.977212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-service-ca\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.977327 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.977232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-oauth-serving-cert\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.979503 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.979480 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-oauth-config\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.979602 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.979569 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-serving-cert\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:12.987531 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:12.987508 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-przlx\" (UniqueName: \"kubernetes.io/projected/bba28cab-660e-43f1-abf1-471e2cc17889-kube-api-access-przlx\") pod \"console-5db4778974-llvgp\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:13.100060 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.100021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:13.279419 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.279375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-sr8qf\" (UID: \"b0cd79c3-d796-4f80-875c-9e08fe208f7a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:13.279579 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:13.279513 2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Aug 04 09:09:13.279632 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:13.279598 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates podName:b0cd79c3-d796-4f80-875c-9e08fe208f7a nodeName:}" failed. No retries permitted until 2026-08-04 09:09:14.279571429 +0000 UTC m=+71.411780941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-sr8qf" (UID: "b0cd79c3-d796-4f80-875c-9e08fe208f7a") : secret "prometheus-operator-admission-webhook-tls" not found Aug 04 09:09:13.362243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.362184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:13.628739 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.628664 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:13.698446 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.698389 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:13.756783 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.756751 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c77485f5-rbgdv"] Aug 04 09:09:13.759638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.759613 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.773504 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.773476 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 04 09:09:13.783453 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.783429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c77485f5-rbgdv"] Aug 04 09:09:13.802712 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.802679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:13.883925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.883850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-serving-cert\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.883925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.883884 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbvft\" (UniqueName: \"kubernetes.io/projected/8af6371e-d13f-45cc-8f8a-2963b1f98aae-kube-api-access-wbvft\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.883925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.883924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-oauth-config\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.884167 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.883940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-service-ca\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.884167 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.883966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-oauth-serving-cert\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.884167 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.883986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-config\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.884167 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.884008 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-trusted-ca-bundle\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985211 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-serving-cert\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbvft\" (UniqueName: \"kubernetes.io/projected/8af6371e-d13f-45cc-8f8a-2963b1f98aae-kube-api-access-wbvft\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-oauth-config\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-service-ca\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-oauth-serving-cert\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-config\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.985398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.985366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-trusted-ca-bundle\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.986210 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.986158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-oauth-serving-cert\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.986356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.986217 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-service-ca\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.986356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.986321 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-trusted-ca-bundle\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.986558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.986532 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-config\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.988036 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.988009 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-serving-cert\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.988152 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.988138 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-oauth-config\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:13.995041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:13.995020 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbvft\" (UniqueName: \"kubernetes.io/projected/8af6371e-d13f-45cc-8f8a-2963b1f98aae-kube-api-access-wbvft\") pod \"console-55c77485f5-rbgdv\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:14.072417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.072383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:14.287641 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.287600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-sr8qf\" (UID: \"b0cd79c3-d796-4f80-875c-9e08fe208f7a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:14.290553 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.290527 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0cd79c3-d796-4f80-875c-9e08fe208f7a-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-sr8qf\" (UID: \"b0cd79c3-d796-4f80-875c-9e08fe208f7a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:14.422281 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.422234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:14.450765 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.450739 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:14.522118 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.522037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:14.609639 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.609578 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db4778974-llvgp"] Aug 04 09:09:14.613740 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:14.613713 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba28cab_660e_43f1_abf1_471e2cc17889.slice/crio-667e85a89617646c78fd3885adfdc0c5b5c7c0c2ad2ff65f0de005e849058680 WatchSource:0}: Error finding container 667e85a89617646c78fd3885adfdc0c5b5c7c0c2ad2ff65f0de005e849058680: Status 404 returned error can't find the container with id 667e85a89617646c78fd3885adfdc0c5b5c7c0c2ad2ff65f0de005e849058680 Aug 04 09:09:14.628403 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.628382 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c77485f5-rbgdv"] Aug 04 09:09:14.631661 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:14.631636 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af6371e_d13f_45cc_8f8a_2963b1f98aae.slice/crio-99d654e637f79271436f721b3bd7a43c144c513ec088872708cbc0e59fe188f5 WatchSource:0}: Error finding container 99d654e637f79271436f721b3bd7a43c144c513ec088872708cbc0e59fe188f5: Status 404 returned error can't find the container with id 99d654e637f79271436f721b3bd7a43c144c513ec088872708cbc0e59fe188f5 Aug 04 09:09:14.649640 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.649576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf"] Aug 04 09:09:14.653750 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:14.653729 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cd79c3_d796_4f80_875c_9e08fe208f7a.slice/crio-fdcf8d80ea087b66c5a9f7c095f1477b6b8db875e3bcdd79fadcf2f6bb0a4a19 WatchSource:0}: Error finding container fdcf8d80ea087b66c5a9f7c095f1477b6b8db875e3bcdd79fadcf2f6bb0a4a19: Status 404 returned error can't find the container with id fdcf8d80ea087b66c5a9f7c095f1477b6b8db875e3bcdd79fadcf2f6bb0a4a19 Aug 04 09:09:14.739711 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.737894 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db4778974-llvgp" event={"ID":"bba28cab-660e-43f1-abf1-471e2cc17889","Type":"ContainerStarted","Data":"667e85a89617646c78fd3885adfdc0c5b5c7c0c2ad2ff65f0de005e849058680"} Aug 04 09:09:14.741329 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.741294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" event={"ID":"b0cd79c3-d796-4f80-875c-9e08fe208f7a","Type":"ContainerStarted","Data":"fdcf8d80ea087b66c5a9f7c095f1477b6b8db875e3bcdd79fadcf2f6bb0a4a19"} Aug 04 09:09:14.742522 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.742495 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c77485f5-rbgdv" event={"ID":"8af6371e-d13f-45cc-8f8a-2963b1f98aae","Type":"ContainerStarted","Data":"99d654e637f79271436f721b3bd7a43c144c513ec088872708cbc0e59fe188f5"} Aug 04 09:09:14.743907 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.743879 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zcjs4" event={"ID":"057e8a4f-5918-497d-9232-1ce627576f54","Type":"ContainerStarted","Data":"653c830b727724484722996764df01920122a256d7f8468319a7865a5f964037"} Aug 04 09:09:14.744003 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.743984 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:09:14.745303 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.745283 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cnb48" event={"ID":"bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc","Type":"ContainerStarted","Data":"370912d04de7db69dc083c17caae4c2c7c46199ef4def572d14fcfeb97d1b180"} Aug 04 09:09:14.762503 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.762451 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zcjs4" podStartSLOduration=65.713460727 podStartE2EDuration="1m10.762434123s" totalImagesPullingTime="5.048973396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:14.761784958 +0000 UTC m=+71.893994503" watchObservedRunningTime="2026-08-04 09:09:14.762434123 +0000 UTC m=+71.894643656" Aug 04 09:09:14.778871 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:14.778829 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cnb48" podStartSLOduration=65.703348788 podStartE2EDuration="1m10.778816896s" totalImagesPullingTime="5.075468108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:14.778649859 +0000 UTC m=+71.910859391" watchObservedRunningTime="2026-08-04 09:09:14.778816896 +0000 UTC m=+71.911026412" Aug 04 09:09:15.749505 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:15.749468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:15.851283 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:15.851222 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:16.758443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:16.758405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" event={"ID":"b0cd79c3-d796-4f80-875c-9e08fe208f7a","Type":"ContainerStarted","Data":"275f2828ceb6faf878180918adf7112a555fbe9cfa4d300ae9224e187b764da5"} Aug 04 09:09:16.758886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:16.758691 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:16.764149 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:16.764123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" Aug 04 09:09:16.774517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:16.774470 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-sr8qf" podStartSLOduration=3.132728636 podStartE2EDuration="4.774452995s" totalImagesPullingTime="1.641724359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:16.77314548 +0000 UTC m=+73.905355009" watchObservedRunningTime="2026-08-04 09:09:16.774452995 +0000 UTC m=+73.906662528" Aug 04 09:09:17.640040 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:17.640003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:17.738737 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:17.738703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:18.005098 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:18.005061 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:18.769045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:18.769007 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db4778974-llvgp" event={"ID":"bba28cab-660e-43f1-abf1-471e2cc17889","Type":"ContainerStarted","Data":"cbbbe7b033b8aedb1bd16b35f95c599e4978b520f96a9b3faf19fccdf4f124cb"} Aug 04 09:09:18.770635 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:18.770615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c77485f5-rbgdv" event={"ID":"8af6371e-d13f-45cc-8f8a-2963b1f98aae","Type":"ContainerStarted","Data":"622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92"} Aug 04 09:09:18.787440 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:18.787403 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db4778974-llvgp" podStartSLOduration=3.5738543849999997 podStartE2EDuration="6.787391288s" totalImagesPullingTime="3.213536903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:18.786978481 +0000 UTC m=+75.919188046" watchObservedRunningTime="2026-08-04 09:09:18.787391288 +0000 UTC m=+75.919600799" Aug 04 09:09:18.806671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:18.806630 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c77485f5-rbgdv" podStartSLOduration=2.617604431 podStartE2EDuration="5.806618335s" totalImagesPullingTime="3.189013904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:18.806072858 +0000 UTC m=+75.938282393" watchObservedRunningTime="2026-08-04 09:09:18.806618335 +0000 UTC m=+75.938827866" Aug 04 09:09:19.305728 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:19.305696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:19.407125 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:19.407094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:20.357209 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:20.357178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:22.430231 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:22.430194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:22.523605 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:22.523577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:22.782602 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:22.782527 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:22.932345 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:22.932310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:23.100415 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:23.100335 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:23.100415 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:23.100390 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:23.105331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:23.105310 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:23.790515 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:23.790486 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:24.017457 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.017427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:24.072540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.072475 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:24.072623 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.072559 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:24.077472 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.077451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:24.113334 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.113306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:24.575751 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.575714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:24.675547 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.675515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:24.793098 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.793071 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:09:24.840835 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:24.840765 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db4778974-llvgp"] Aug 04 09:09:25.127273 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:25.127175 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:25.228310 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:25.228188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:26.144111 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.144075 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:26.240990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.240956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:26.903979 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.903947 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:26.930629 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.930601 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq"] Aug 04 09:09:26.936498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.936474 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:26.939199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.939178 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Aug 04 09:09:26.939199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.939188 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 09:09:26.939356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.939197 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 09:09:26.939356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.939242 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-h4prc\"" Aug 04 09:09:26.939356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.939187 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Aug 04 09:09:26.939356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.939287 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 09:09:26.949124 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.949103 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq"] Aug 04 09:09:26.980904 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.980876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5nd6\" (UniqueName: \"kubernetes.io/projected/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-kube-api-access-b5nd6\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:26.981014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.980917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:26.981014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.980944 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:26.981091 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:26.981074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.000921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.000897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.048758 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.048735 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qrqkf"] Aug 04 09:09:27.052620 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.052603 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.055935 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.055886 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 09:09:27.056046 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.055891 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lhsjg\"" Aug 04 09:09:27.056241 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.056222 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 09:09:27.056339 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.056276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 09:09:27.056707 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.056690 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6"] Aug 04 09:09:27.060414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.060398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.063238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.063220 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7vx7x\"" Aug 04 09:09:27.063238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.063229 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 04 09:09:27.063381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.063299 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 04 09:09:27.063381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.063315 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 04 09:09:27.074574 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.074556 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6"] Aug 04 09:09:27.081676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.081645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.081778 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.081685 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5nd6\" (UniqueName: \"kubernetes.io/projected/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-kube-api-access-b5nd6\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.081778 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.081707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.081778 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.081727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.082434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.082409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.083966 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.083946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.084060 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.083985 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.091775 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.091757 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5nd6\" (UniqueName: \"kubernetes.io/projected/c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52-kube-api-access-b5nd6\") pod \"openshift-state-metrics-6bb4d7b4dd-tgftq\" (UID: \"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.107925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.107898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.182654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/019cf2ad-7780-423b-a94f-2fd9badc4649-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.182654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.182654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-textfile\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56l9g\" (UniqueName: \"kubernetes.io/projected/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-api-access-56l9g\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182783 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/019cf2ad-7780-423b-a94f-2fd9badc4649-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb82w\" (UniqueName: \"kubernetes.io/projected/c61bb269-a513-45f4-9926-aa19d9fd246d-kube-api-access-zb82w\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-wtmp\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c61bb269-a513-45f4-9926-aa19d9fd246d-metrics-client-ca\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182944 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-tls\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182961 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.182985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-root\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.183324 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.182994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-sys\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.200852 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.200828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.245073 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.245048 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" Aug 04 09:09:27.283901 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.283867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/019cf2ad-7780-423b-a94f-2fd9badc4649-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.284034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.283910 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb82w\" (UniqueName: \"kubernetes.io/projected/c61bb269-a513-45f4-9926-aa19d9fd246d-kube-api-access-zb82w\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.283960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.284034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.283991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-wtmp\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c61bb269-a513-45f4-9926-aa19d9fd246d-metrics-client-ca\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284072 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-tls\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-root\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-sys\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/019cf2ad-7780-423b-a94f-2fd9badc4649-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.284271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-textfile\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284635 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56l9g\" (UniqueName: \"kubernetes.io/projected/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-api-access-56l9g\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.284635 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284798 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284770 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-root\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.284885 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:27.284805 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Aug 04 09:09:27.284885 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:27.284867 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-tls podName:019cf2ad-7780-423b-a94f-2fd9badc4649 nodeName:}" failed. No retries permitted until 2026-08-04 09:09:27.784845963 +0000 UTC m=+84.917055474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-j78v6" (UID: "019cf2ad-7780-423b-a94f-2fd9badc4649") : secret "kube-state-metrics-tls" not found Aug 04 09:09:27.285000 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284926 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-textfile\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.285000 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.284964 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/019cf2ad-7780-423b-a94f-2fd9badc4649-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.285203 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.285166 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-sys\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.285334 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:27.285300 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 09:09:27.285414 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:27.285392 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-tls podName:c61bb269-a513-45f4-9926-aa19d9fd246d nodeName:}" failed. No retries permitted until 2026-08-04 09:09:27.785372608 +0000 UTC m=+84.917582130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-tls") pod "node-exporter-qrqkf" (UID: "c61bb269-a513-45f4-9926-aa19d9fd246d") : secret "node-exporter-tls" not found Aug 04 09:09:27.285414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.285172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-wtmp\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.285414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.285405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c61bb269-a513-45f4-9926-aa19d9fd246d-metrics-client-ca\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.285414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.285434 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.285670 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.285493 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.285670 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.285615 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/019cf2ad-7780-423b-a94f-2fd9badc4649-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.287055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.287036 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.287512 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.287495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.293349 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.293314 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb82w\" (UniqueName: \"kubernetes.io/projected/c61bb269-a513-45f4-9926-aa19d9fd246d-kube-api-access-zb82w\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.293761 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.293738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56l9g\" (UniqueName: \"kubernetes.io/projected/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-api-access-56l9g\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.318128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.318102 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.374907 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.374878 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq"] Aug 04 09:09:27.378333 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:27.378305 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b9cd19_8aaa_458c_8d1c_ad9cd6698e52.slice/crio-8fca63bf8ad4d72e38155f5116d70c40e14b3aa47501e717c43237f04a7d8310 WatchSource:0}: Error finding container 8fca63bf8ad4d72e38155f5116d70c40e14b3aa47501e717c43237f04a7d8310: Status 404 returned error can't find the container with id 8fca63bf8ad4d72e38155f5116d70c40e14b3aa47501e717c43237f04a7d8310 Aug 04 09:09:27.381085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.381063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.459823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.459795 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.711710 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.711630 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:27.787959 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.787925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.788119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.787970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-tls\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.791936 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.791102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c61bb269-a513-45f4-9926-aa19d9fd246d-node-exporter-tls\") pod \"node-exporter-qrqkf\" (UID: \"c61bb269-a513-45f4-9926-aa19d9fd246d\") " pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.793157 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.793137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/019cf2ad-7780-423b-a94f-2fd9badc4649-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-j78v6\" (UID: \"019cf2ad-7780-423b-a94f-2fd9badc4649\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.800427 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.800400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" event={"ID":"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52","Type":"ContainerStarted","Data":"3530fa7b9105d1d05faf64a103585d324dd2f1f4f08a4ba1c41ef18008537248"} Aug 04 09:09:27.800523 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.800474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" event={"ID":"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52","Type":"ContainerStarted","Data":"dbc8f5da9ce7a74dc1e1216dd812ef88760f3f4811dc239371ebffedd14e187b"} Aug 04 09:09:27.800523 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.800498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" event={"ID":"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52","Type":"ContainerStarted","Data":"8fca63bf8ad4d72e38155f5116d70c40e14b3aa47501e717c43237f04a7d8310"} Aug 04 09:09:27.961520 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.961488 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qrqkf" Aug 04 09:09:27.967688 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:27.967671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" Aug 04 09:09:27.971665 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:27.971638 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61bb269_a513_45f4_9926_aa19d9fd246d.slice/crio-bdae35c2732c67eaf46ba490a0d6ff2d8ba1dcef72ccbe33c1cba2f558d663ca WatchSource:0}: Error finding container bdae35c2732c67eaf46ba490a0d6ff2d8ba1dcef72ccbe33c1cba2f558d663ca: Status 404 returned error can't find the container with id bdae35c2732c67eaf46ba490a0d6ff2d8ba1dcef72ccbe33c1cba2f558d663ca Aug 04 09:09:28.075931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.075895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:28.094525 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.094496 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6"] Aug 04 09:09:28.097775 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:28.097740 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019cf2ad_7780_423b_a94f_2fd9badc4649.slice/crio-cfbfc3a3672728a72d17883252529505881810a5312aa1614e5d367b5fa85428 WatchSource:0}: Error finding container cfbfc3a3672728a72d17883252529505881810a5312aa1614e5d367b5fa85428: Status 404 returned error can't find the container with id cfbfc3a3672728a72d17883252529505881810a5312aa1614e5d367b5fa85428 Aug 04 09:09:28.184475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.184429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:28.201099 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.201077 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:09:28.205022 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.205001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.208528 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.208509 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 04 09:09:28.208831 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.208808 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 04 09:09:28.208926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.208836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 04 09:09:28.208984 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.208817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h7mdh\"" Aug 04 09:09:28.209132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.209104 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 04 09:09:28.209208 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.209162 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 04 09:09:28.209208 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.209180 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 04 09:09:28.209358 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.209336 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 04 09:09:28.209462 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.209431 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 04 09:09:28.209517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.209479 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 04 09:09:28.225833 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.225781 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:09:28.292923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.292884 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.292931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45219cfe-41d1-4a96-933a-074bd8063e0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.292986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-web-config\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45219cfe-41d1-4a96-933a-074bd8063e0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293144 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45219cfe-41d1-4a96-933a-074bd8063e0b-config-out\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45219cfe-41d1-4a96-933a-074bd8063e0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7k4z\" (UniqueName: \"kubernetes.io/projected/45219cfe-41d1-4a96-933a-074bd8063e0b-kube-api-access-g7k4z\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.293516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.293462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45219cfe-41d1-4a96-933a-074bd8063e0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394009 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.393962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45219cfe-41d1-4a96-933a-074bd8063e0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394009 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45219cfe-41d1-4a96-933a-074bd8063e0b-config-out\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394332 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45219cfe-41d1-4a96-933a-074bd8063e0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394332 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7k4z\" (UniqueName: \"kubernetes.io/projected/45219cfe-41d1-4a96-933a-074bd8063e0b-kube-api-access-g7k4z\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394523 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394523 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45219cfe-41d1-4a96-933a-074bd8063e0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394614 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394614 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:28.394537 2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Aug 04 09:09:28.394614 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45219cfe-41d1-4a96-933a-074bd8063e0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394614 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-web-config\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.394614 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:28.394609 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-main-tls podName:45219cfe-41d1-4a96-933a-074bd8063e0b nodeName:}" failed. No retries permitted until 2026-08-04 09:09:28.894584716 +0000 UTC m=+86.026794248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "45219cfe-41d1-4a96-933a-074bd8063e0b") : secret "alertmanager-main-tls" not found Aug 04 09:09:28.394847 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.395797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.394934 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45219cfe-41d1-4a96-933a-074bd8063e0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.395797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.395437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45219cfe-41d1-4a96-933a-074bd8063e0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.396305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.396224 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45219cfe-41d1-4a96-933a-074bd8063e0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.397676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.397643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.397789 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.397704 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-web-config\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.398441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.398369 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45219cfe-41d1-4a96-933a-074bd8063e0b-config-out\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.398548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.398518 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.398619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.398585 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45219cfe-41d1-4a96-933a-074bd8063e0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.398844 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.398807 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.398963 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.398946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.399949 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.399921 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.404405 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.404383 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7k4z\" (UniqueName: \"kubernetes.io/projected/45219cfe-41d1-4a96-933a-074bd8063e0b-kube-api-access-g7k4z\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.806794 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.806752 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" event={"ID":"019cf2ad-7780-423b-a94f-2fd9badc4649","Type":"ContainerStarted","Data":"cfbfc3a3672728a72d17883252529505881810a5312aa1614e5d367b5fa85428"} Aug 04 09:09:28.809592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.809512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" event={"ID":"c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52","Type":"ContainerStarted","Data":"3228fc7a58e65b1d5239ce15165d34548c77d38e96585a0d52d185aad5885de3"} Aug 04 09:09:28.810901 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.810849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrqkf" event={"ID":"c61bb269-a513-45f4-9926-aa19d9fd246d","Type":"ContainerStarted","Data":"bdae35c2732c67eaf46ba490a0d6ff2d8ba1dcef72ccbe33c1cba2f558d663ca"} Aug 04 09:09:28.833586 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.833486 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-tgftq" podStartSLOduration=1.743070233 podStartE2EDuration="2.833466452s" totalImagesPullingTime="1.090396219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:28.831346181 +0000 UTC m=+85.963555714" watchObservedRunningTime="2026-08-04 09:09:28.833466452 +0000 UTC m=+85.965675985" Aug 04 09:09:28.899151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.899109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.902240 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.902211 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45219cfe-41d1-4a96-933a-074bd8063e0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45219cfe-41d1-4a96-933a-074bd8063e0b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:28.935157 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:28.935119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:29.033444 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.033404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:29.116331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.116261 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:09:29.610662 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.610609 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:09:29.614294 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:29.614264 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45219cfe_41d1_4a96_933a_074bd8063e0b.slice/crio-120c0c012d4d1a0dba66ac1e8062e452062b9af7a18c9c883d24b6f4ef886f3c WatchSource:0}: Error finding container 120c0c012d4d1a0dba66ac1e8062e452062b9af7a18c9c883d24b6f4ef886f3c: Status 404 returned error can't find the container with id 120c0c012d4d1a0dba66ac1e8062e452062b9af7a18c9c883d24b6f4ef886f3c Aug 04 09:09:29.817204 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.817148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" event={"ID":"019cf2ad-7780-423b-a94f-2fd9badc4649","Type":"ContainerStarted","Data":"354b2e24d4e9e46f31e3b16ab89f3b44cce9b0cb61b254178aee4ad82ddd444b"} Aug 04 09:09:29.817402 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.817245 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" event={"ID":"019cf2ad-7780-423b-a94f-2fd9badc4649","Type":"ContainerStarted","Data":"05789cf8b194f1b3eb54aafb821701c7c7b7799872570d2d4a0c9074d4e5d89e"} Aug 04 09:09:29.817402 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.817292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" event={"ID":"019cf2ad-7780-423b-a94f-2fd9badc4649","Type":"ContainerStarted","Data":"e0c8651e854cff4a84ddafb3c3741b26fc693254d354b802384cd73d2d26ce8a"} Aug 04 09:09:29.818542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.818513 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"120c0c012d4d1a0dba66ac1e8062e452062b9af7a18c9c883d24b6f4ef886f3c"} Aug 04 09:09:29.820313 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.820287 2562 generic.go:356] "Generic (PLEG): container finished" podID="c61bb269-a513-45f4-9926-aa19d9fd246d" containerID="08f1de5b62a4d13926dbf25b724d35346cbe004ca5554b0a023bbe1ca29e0848" exitCode=0 Aug 04 09:09:29.820420 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.820376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrqkf" event={"ID":"c61bb269-a513-45f4-9926-aa19d9fd246d","Type":"ContainerDied","Data":"08f1de5b62a4d13926dbf25b724d35346cbe004ca5554b0a023bbe1ca29e0848"} Aug 04 09:09:29.841538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.841500 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-j78v6" podStartSLOduration=1.4724412230000001 podStartE2EDuration="2.841484387s" totalImagesPullingTime="1.369043164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:29.839392425 +0000 UTC m=+86.971601957" watchObservedRunningTime="2026-08-04 09:09:29.841484387 +0000 UTC m=+86.973693918" Aug 04 09:09:29.967764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:29.967679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:30.031995 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.031959 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c88ff454b-bjmcj"] Aug 04 09:09:30.035446 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.035429 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.038185 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038160 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 04 09:09:30.038320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038192 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 04 09:09:30.038320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038209 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 04 09:09:30.038615 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038599 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 04 09:09:30.038679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038599 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bfl59\"" Aug 04 09:09:30.038776 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038759 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bvvq5ach4ng98\"" Aug 04 09:09:30.038851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.038837 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 04 09:09:30.048189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.048166 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c88ff454b-bjmcj"] Aug 04 09:09:30.051575 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.051552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:30.108222 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de61d19-02e9-492b-b2c9-80c27f8937c4-metrics-client-ca\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mndzk\" (UniqueName: \"kubernetes.io/projected/1de61d19-02e9-492b-b2c9-80c27f8937c4-kube-api-access-mndzk\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108349 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-tls\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.108449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.108422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-grpc-tls\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.146416 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.146392 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:30.208736 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mndzk\" (UniqueName: \"kubernetes.io/projected/1de61d19-02e9-492b-b2c9-80c27f8937c4-kube-api-access-mndzk\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.208736 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.208886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.208886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.208886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-tls\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.208886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.209080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-grpc-tls\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.209080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.208935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de61d19-02e9-492b-b2c9-80c27f8937c4-metrics-client-ca\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.209947 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.209919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de61d19-02e9-492b-b2c9-80c27f8937c4-metrics-client-ca\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.211745 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.211722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-tls\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.211858 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.211745 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.211900 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.211867 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.211900 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.211892 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.211961 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.211950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-grpc-tls\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.211992 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.211954 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1de61d19-02e9-492b-b2c9-80c27f8937c4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.217957 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.217909 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mndzk\" (UniqueName: \"kubernetes.io/projected/1de61d19-02e9-492b-b2c9-80c27f8937c4-kube-api-access-mndzk\") pod \"thanos-querier-7c88ff454b-bjmcj\" (UID: \"1de61d19-02e9-492b-b2c9-80c27f8937c4\") " pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.264552 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.264521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:30.344913 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.344886 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:30.475774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.475698 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c88ff454b-bjmcj"] Aug 04 09:09:30.479608 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:30.479570 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de61d19_02e9_492b_b2c9_80c27f8937c4.slice/crio-8d1dfffe5087638084eed273bcd2184ca8331e370c6a0096dff7ca4273dd0008 WatchSource:0}: Error finding container 8d1dfffe5087638084eed273bcd2184ca8331e370c6a0096dff7ca4273dd0008: Status 404 returned error can't find the container with id 8d1dfffe5087638084eed273bcd2184ca8331e370c6a0096dff7ca4273dd0008 Aug 04 09:09:30.826631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.826550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"8d1dfffe5087638084eed273bcd2184ca8331e370c6a0096dff7ca4273dd0008"} Aug 04 09:09:30.828985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.828949 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrqkf" event={"ID":"c61bb269-a513-45f4-9926-aa19d9fd246d","Type":"ContainerStarted","Data":"fb0a93c95c95dcd80f4982a28f39e78411827270f51290598a5ef8bbb8a4e3fc"} Aug 04 09:09:30.829129 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:30.829010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrqkf" event={"ID":"c61bb269-a513-45f4-9926-aa19d9fd246d","Type":"ContainerStarted","Data":"8cfbaccdf2791cf55d8512f63d21f96605c0e1baf34c8bf3c1db27f3072a6e70"} Aug 04 09:09:31.833737 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.833678 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qrqkf" podStartSLOduration=3.254144295 podStartE2EDuration="4.833653085s" totalImagesPullingTime="1.492526146s" totalInitContainerRuntime="86.982644ms" isStatefulPod=true podCreationTimestamp="2026-08-04 09:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:30.851608553 +0000 UTC m=+87.983818086" watchObservedRunningTime="2026-08-04 09:09:31.833653085 +0000 UTC m=+88.965862617" Aug 04 09:09:31.834169 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.833907 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fcfd957b5-l9xv8"] Aug 04 09:09:31.836332 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.836301 2562 generic.go:356] "Generic (PLEG): container finished" podID="45219cfe-41d1-4a96-933a-074bd8063e0b" containerID="5c242924a5c9efc95be9123bcbb234e85551049d05be16bd5c52d4a10ebc46da" exitCode=0 Aug 04 09:09:31.844131 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.844104 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:31.849778 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.849753 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb"] Aug 04 09:09:31.849878 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.849860 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.863795 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.863770 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcfd957b5-l9xv8"] Aug 04 09:09:31.863897 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.863807 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb"] Aug 04 09:09:31.863897 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.863886 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:31.864017 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.863916 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerDied","Data":"5c242924a5c9efc95be9123bcbb234e85551049d05be16bd5c52d4a10ebc46da"} Aug 04 09:09:31.866279 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.866237 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Aug 04 09:09:31.866370 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.866239 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jzjgr\"" Aug 04 09:09:31.914518 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.914481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:31.926293 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-oauth-config\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.926454 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-oauth-serving-cert\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.927027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-serving-cert\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.927027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-service-ca\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.927027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8730d5cb-21cf-43fe-b552-4aecc6feb94d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:31.927027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-trusted-ca-bundle\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.927027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf56k\" (UniqueName: \"kubernetes.io/projected/8730d5cb-21cf-43fe-b552-4aecc6feb94d-kube-api-access-kf56k\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:31.927027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.926815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwkns\" (UniqueName: \"kubernetes.io/projected/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-kube-api-access-gwkns\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:31.927403 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:31.927244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-config\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.005792 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.005761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:32.028137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-service-ca\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8730d5cb-21cf-43fe-b552-4aecc6feb94d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:32.028295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-trusted-ca-bundle\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf56k\" (UniqueName: \"kubernetes.io/projected/8730d5cb-21cf-43fe-b552-4aecc6feb94d-kube-api-access-kf56k\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:32.028295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwkns\" (UniqueName: \"kubernetes.io/projected/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-kube-api-access-gwkns\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028295 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:32.028276 2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Aug 04 09:09:32.028572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-config\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028324 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-oauth-config\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028572 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:09:32.028348 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8730d5cb-21cf-43fe-b552-4aecc6feb94d-monitoring-plugin-cert podName:8730d5cb-21cf-43fe-b552-4aecc6feb94d nodeName:}" failed. No retries permitted until 2026-08-04 09:09:32.528324608 +0000 UTC m=+89.660534137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8730d5cb-21cf-43fe-b552-4aecc6feb94d-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-4zbwb" (UID: "8730d5cb-21cf-43fe-b552-4aecc6feb94d") : secret "monitoring-plugin-cert" not found Aug 04 09:09:32.028572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-oauth-serving-cert\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-serving-cert\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.028965 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.028942 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-service-ca\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.029070 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.029045 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-trusted-ca-bundle\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.029332 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.029313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-oauth-serving-cert\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.029415 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.029365 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-config\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.030903 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.030885 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-serving-cert\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.030982 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.030960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-oauth-config\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.040097 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.040074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwkns\" (UniqueName: \"kubernetes.io/projected/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-kube-api-access-gwkns\") pod \"console-5fcfd957b5-l9xv8\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.040443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.040421 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf56k\" (UniqueName: \"kubernetes.io/projected/8730d5cb-21cf-43fe-b552-4aecc6feb94d-kube-api-access-kf56k\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:32.103496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.103403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:32.161109 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.161079 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:32.204099 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.204056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:32.264400 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.264366 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-786fb7c68f-rsss9"] Aug 04 09:09:32.278554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.278531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.281404 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.281381 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Aug 04 09:09:32.281404 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.281383 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Aug 04 09:09:32.285614 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.285590 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Aug 04 09:09:32.285853 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.285832 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Aug 04 09:09:32.290089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.289508 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w886h\"" Aug 04 09:09:32.290089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.289684 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Aug 04 09:09:32.291625 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.291601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-786fb7c68f-rsss9"] Aug 04 09:09:32.292893 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.292870 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Aug 04 09:09:32.432528 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-secret-telemeter-client\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-telemeter-client-tls\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-federate-client-tls\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-metrics-client-ca\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432966 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvnq7\" (UniqueName: \"kubernetes.io/projected/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-kube-api-access-zvnq7\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.432966 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.432749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-serving-certs-ca-bundle\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.533429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.533588 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-secret-telemeter-client\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.533689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.533771 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8730d5cb-21cf-43fe-b552-4aecc6feb94d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:32.533830 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-telemeter-client-tls\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.533881 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-federate-client-tls\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.533881 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.533862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-metrics-client-ca\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.534037 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.534005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvnq7\" (UniqueName: \"kubernetes.io/projected/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-kube-api-access-zvnq7\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.534188 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.534082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-serving-certs-ca-bundle\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.534727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.534399 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.534727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.534513 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-metrics-client-ca\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.534727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.534699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-serving-certs-ca-bundle\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.536718 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.536692 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.536789 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.536747 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8730d5cb-21cf-43fe-b552-4aecc6feb94d-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-4zbwb\" (UID: \"8730d5cb-21cf-43fe-b552-4aecc6feb94d\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:32.536906 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.536869 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-secret-telemeter-client\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.537469 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.537450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-federate-client-tls\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.537898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.537878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-telemeter-client-tls\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.551042 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.551019 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvnq7\" (UniqueName: \"kubernetes.io/projected/4fe431d7-ee2c-4953-9c9f-315fd661dc8b-kube-api-access-zvnq7\") pod \"telemeter-client-786fb7c68f-rsss9\" (UID: \"4fe431d7-ee2c-4953-9c9f-315fd661dc8b\") " pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.591516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.591478 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" Aug 04 09:09:32.776397 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.776369 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:32.803042 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.802689 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-786fb7c68f-rsss9"] Aug 04 09:09:32.807330 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:32.807296 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe431d7_ee2c_4953_9c9f_315fd661dc8b.slice/crio-780bd2a2cdfa79915ea15d184d998100d174bd14026d2708915bbbc7227c4756 WatchSource:0}: Error finding container 780bd2a2cdfa79915ea15d184d998100d174bd14026d2708915bbbc7227c4756: Status 404 returned error can't find the container with id 780bd2a2cdfa79915ea15d184d998100d174bd14026d2708915bbbc7227c4756 Aug 04 09:09:32.826158 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.826102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcfd957b5-l9xv8"] Aug 04 09:09:32.833186 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:32.833159 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf656a6c0_67ce_4d3e_ae73_a0ecaac83cce.slice/crio-3525f41f6e2dffc732e10cd295457c69b9f9feadf9e6569a29dd281e4c52f7ef WatchSource:0}: Error finding container 3525f41f6e2dffc732e10cd295457c69b9f9feadf9e6569a29dd281e4c52f7ef: Status 404 returned error can't find the container with id 3525f41f6e2dffc732e10cd295457c69b9f9feadf9e6569a29dd281e4c52f7ef Aug 04 09:09:32.854551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.854527 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"f1f1dfda0a5f6c09e96321a25acf80a1ed19162b86d91fc7d13167412866341c"} Aug 04 09:09:32.856299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.856276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" event={"ID":"4fe431d7-ee2c-4953-9c9f-315fd661dc8b","Type":"ContainerStarted","Data":"780bd2a2cdfa79915ea15d184d998100d174bd14026d2708915bbbc7227c4756"} Aug 04 09:09:32.857747 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.857722 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfd957b5-l9xv8" event={"ID":"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce","Type":"ContainerStarted","Data":"3525f41f6e2dffc732e10cd295457c69b9f9feadf9e6569a29dd281e4c52f7ef"} Aug 04 09:09:32.933566 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:32.933512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb"] Aug 04 09:09:32.937825 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:32.937753 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8730d5cb_21cf_43fe_b552_4aecc6feb94d.slice/crio-9e894eb4db7c0fac9a9b396482acf6e501a5104217d54e2e3c1f50bb7f5871db WatchSource:0}: Error finding container 9e894eb4db7c0fac9a9b396482acf6e501a5104217d54e2e3c1f50bb7f5871db: Status 404 returned error can't find the container with id 9e894eb4db7c0fac9a9b396482acf6e501a5104217d54e2e3c1f50bb7f5871db Aug 04 09:09:33.277843 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:33.277738 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:33.866013 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:33.865973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"a55d25745e4070947e761a83473cef08b501064e3555a320149a12d5393cfcae"} Aug 04 09:09:33.866490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:33.866056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"9246a42484c01c1c38da86325bae136138882400d57b7cf5ccefae4cc53579b8"} Aug 04 09:09:33.868202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:33.868171 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfd957b5-l9xv8" event={"ID":"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce","Type":"ContainerStarted","Data":"77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83"} Aug 04 09:09:33.870117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:33.870090 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" event={"ID":"8730d5cb-21cf-43fe-b552-4aecc6feb94d","Type":"ContainerStarted","Data":"9e894eb4db7c0fac9a9b396482acf6e501a5104217d54e2e3c1f50bb7f5871db"} Aug 04 09:09:33.891233 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:33.891171 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fcfd957b5-l9xv8" podStartSLOduration=2.8911488800000003 podStartE2EDuration="2.89114888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:33.890659961 +0000 UTC m=+91.022869493" watchObservedRunningTime="2026-08-04 09:09:33.89114888 +0000 UTC m=+91.023358415" Aug 04 09:09:35.882241 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.882201 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"496b116e8c68e5a2ebd5c4507ae9a76e5ebd438a0ed904278b1455509e31c933"} Aug 04 09:09:35.882716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.882308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"52f9e6875592fcce4b0bd5db04dac18a4cdfd197f3eb7035316cd42100b43f50"} Aug 04 09:09:35.882716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.882342 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"59925e9f9e861d742f2b7c251b757697ab65ea9c6f0351a4ece0f3c201093fd7"} Aug 04 09:09:35.882716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.882369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"c0cf58cb17771aec84481a2fa904acb51cbf793f4c17ea4b967c0b0dc00bd7bc"} Aug 04 09:09:35.882716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.882395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"8ec391e83ee7e18f80fbd433f70c1f0995ed9ba99c4d167054ab4224be13ae4d"} Aug 04 09:09:35.882716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.882429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45219cfe-41d1-4a96-933a-074bd8063e0b","Type":"ContainerStarted","Data":"167473dbd63eefb6430ef74bdaedc390966a13d247f908d8d6b1bd7c520fe6f7"} Aug 04 09:09:35.884907 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.884879 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"2a634522a8bb19d90fcc72536c665bec4dfce518836273dcd694993e20a03536"} Aug 04 09:09:35.885045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.884935 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"3e604ceebd1b4ee62dac9f2db696fb11ae2a2fa8b878640bb54a65ddbeddfda8"} Aug 04 09:09:35.885045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.884962 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" event={"ID":"1de61d19-02e9-492b-b2c9-80c27f8937c4","Type":"ContainerStarted","Data":"ed633f1f00c231aef8d8ac884258fdda79803f19247956601ccd3425e7cd6014"} Aug 04 09:09:35.885045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.885004 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:35.886945 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.886920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" event={"ID":"4fe431d7-ee2c-4953-9c9f-315fd661dc8b","Type":"ContainerStarted","Data":"5a0d6d744088c284aab84031f148320d202bcc2e41f09542da0cd96159833262"} Aug 04 09:09:35.887070 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.886976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" event={"ID":"4fe431d7-ee2c-4953-9c9f-315fd661dc8b","Type":"ContainerStarted","Data":"f3a619f147019a829049e3670acfd606efcc53518380ed0bcd2dc5a38e455c0c"} Aug 04 09:09:35.887070 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.887006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" event={"ID":"4fe431d7-ee2c-4953-9c9f-315fd661dc8b","Type":"ContainerStarted","Data":"8121a898dde3364f94f19d53ef1072967feb7b7ee061f6f7e18f106a7ebda65c"} Aug 04 09:09:35.888436 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.888413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" event={"ID":"8730d5cb-21cf-43fe-b552-4aecc6feb94d","Type":"ContainerStarted","Data":"885732b59d4e320ca4dd1a00aed6e5d580943f0697b59918104fae8e5e395ac0"} Aug 04 09:09:35.888599 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.888584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:35.893660 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.893639 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" Aug 04 09:09:35.913126 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.913068 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.036612392 podStartE2EDuration="7.913055876s" totalImagesPullingTime="4.851111899s" totalInitContainerRuntime="25.331585ms" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:35.910999901 +0000 UTC m=+93.043209433" watchObservedRunningTime="2026-08-04 09:09:35.913055876 +0000 UTC m=+93.045265407" Aug 04 09:09:35.937379 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.937331 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" podStartSLOduration=1.663798072 podStartE2EDuration="5.937317009s" totalImagesPullingTime="4.273518937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:35.935422303 +0000 UTC m=+93.067631836" watchObservedRunningTime="2026-08-04 09:09:35.937317009 +0000 UTC m=+93.069526541" Aug 04 09:09:35.952202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.952164 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-4zbwb" podStartSLOduration=2.895605119 podStartE2EDuration="4.952150879s" totalImagesPullingTime="2.05654576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:35.95105288 +0000 UTC m=+93.083262412" watchObservedRunningTime="2026-08-04 09:09:35.952150879 +0000 UTC m=+93.084360410" Aug 04 09:09:35.971526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:35.971491 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-786fb7c68f-rsss9" podStartSLOduration=1.7395247999999999 podStartE2EDuration="3.971478817s" totalImagesPullingTime="2.231954017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:35.970244437 +0000 UTC m=+93.102453968" watchObservedRunningTime="2026-08-04 09:09:35.971478817 +0000 UTC m=+93.103688349" Aug 04 09:09:36.063200 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.063171 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcfd957b5-l9xv8"] Aug 04 09:09:36.093452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.093422 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79bf8b8887-cc5vl"] Aug 04 09:09:36.096780 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.096758 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.112307 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.112282 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79bf8b8887-cc5vl"] Aug 04 09:09:36.170803 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.170776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-oauth-config\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.170926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.170808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtkbz\" (UniqueName: \"kubernetes.io/projected/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-kube-api-access-vtkbz\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.170926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.170827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-oauth-serving-cert\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.170926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.170905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-config\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.171039 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.170933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-trusted-ca-bundle\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.171039 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.170957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-serving-cert\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.171039 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.171020 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-service-ca\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.272355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272321 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-oauth-config\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.272355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtkbz\" (UniqueName: \"kubernetes.io/projected/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-kube-api-access-vtkbz\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.272538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-oauth-serving-cert\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.272538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-config\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.272538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-trusted-ca-bundle\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.272914 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-serving-cert\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.273013 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.272990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-service-ca\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.273629 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.273605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-oauth-serving-cert\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.273758 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.273716 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-service-ca\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.274505 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.274481 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-config\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.279321 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.279294 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-oauth-config\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.279495 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.279464 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-serving-cert\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.279681 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.279659 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-trusted-ca-bundle\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.281481 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.281463 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtkbz\" (UniqueName: \"kubernetes.io/projected/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-kube-api-access-vtkbz\") pod \"console-79bf8b8887-cc5vl\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.405500 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.405454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:36.538605 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.538583 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79bf8b8887-cc5vl"] Aug 04 09:09:36.540519 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:09:36.540491 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ceb4474_3ca0_434f_8ee8_d8793f5f08d5.slice/crio-f955ccd3af1302ce32d3e78aafd07b412eb61c5efce66e38d099d810b2f1980e WatchSource:0}: Error finding container f955ccd3af1302ce32d3e78aafd07b412eb61c5efce66e38d099d810b2f1980e: Status 404 returned error can't find the container with id f955ccd3af1302ce32d3e78aafd07b412eb61c5efce66e38d099d810b2f1980e Aug 04 09:09:36.894906 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.894865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf8b8887-cc5vl" event={"ID":"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5","Type":"ContainerStarted","Data":"ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2"} Aug 04 09:09:36.895329 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.894924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf8b8887-cc5vl" event={"ID":"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5","Type":"ContainerStarted","Data":"f955ccd3af1302ce32d3e78aafd07b412eb61c5efce66e38d099d810b2f1980e"} Aug 04 09:09:36.913587 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:36.913541 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79bf8b8887-cc5vl" podStartSLOduration=0.913526643 podStartE2EDuration="913.526643ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:09:36.912034722 +0000 UTC m=+94.044244256" watchObservedRunningTime="2026-08-04 09:09:36.913526643 +0000 UTC m=+94.045736174" Aug 04 09:09:39.193022 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:39.192992 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4tpsh_e4ae14d4-e5c9-43d0-89ef-22edc19a7255/serve-healthcheck-canary/0.log" Aug 04 09:09:41.141361 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:41.141325 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:41.202767 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:41.202733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:41.900950 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:41.900924 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c88ff454b-bjmcj" Aug 04 09:09:42.161476 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:42.161394 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:09:42.520452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:42.520363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:42.621559 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:42.621524 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:45.756667 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:45.756635 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zcjs4" Aug 04 09:09:46.406418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:46.406385 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:46.406670 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:46.406434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:46.411004 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:46.410983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:46.938894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:46.938863 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:09:46.988479 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:46.988454 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c77485f5-rbgdv"] Aug 04 09:09:47.173100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:47.173071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:47.285760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:47.285674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:50.879928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:50.879850 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:50.983720 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:50.983685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:51.043739 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:51.043707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:09:51.814037 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:51.813987 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5db4778974-llvgp" podUID="bba28cab-660e-43f1-abf1-471e2cc17889" containerName="console" containerID="cri-o://cbbbe7b033b8aedb1bd16b35f95c599e4978b520f96a9b3faf19fccdf4f124cb" gracePeriod=15 Aug 04 09:09:51.958922 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:51.958887 2562 generic.go:356] "Generic (PLEG): container finished" podID="bba28cab-660e-43f1-abf1-471e2cc17889" containerID="cbbbe7b033b8aedb1bd16b35f95c599e4978b520f96a9b3faf19fccdf4f124cb" exitCode=0 Aug 04 09:09:51.959295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:51.958954 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db4778974-llvgp" event={"ID":"bba28cab-660e-43f1-abf1-471e2cc17889","Type":"ContainerDied","Data":"cbbbe7b033b8aedb1bd16b35f95c599e4978b520f96a9b3faf19fccdf4f124cb"} Aug 04 09:09:52.098179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.098154 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:52.104149 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104126 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-przlx\" (UniqueName: \"kubernetes.io/projected/bba28cab-660e-43f1-abf1-471e2cc17889-kube-api-access-przlx\") pod \"bba28cab-660e-43f1-abf1-471e2cc17889\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " Aug 04 09:09:52.104284 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104161 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-oauth-config\") pod \"bba28cab-660e-43f1-abf1-471e2cc17889\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " Aug 04 09:09:52.104284 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104189 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-serving-cert\") pod \"bba28cab-660e-43f1-abf1-471e2cc17889\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " Aug 04 09:09:52.104284 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104207 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-service-ca\") pod \"bba28cab-660e-43f1-abf1-471e2cc17889\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " Aug 04 09:09:52.104444 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104313 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-oauth-serving-cert\") pod \"bba28cab-660e-43f1-abf1-471e2cc17889\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " Aug 04 09:09:52.104444 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104368 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-console-config\") pod \"bba28cab-660e-43f1-abf1-471e2cc17889\" (UID: \"bba28cab-660e-43f1-abf1-471e2cc17889\") " Aug 04 09:09:52.104810 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104776 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-service-ca" (OuterVolumeSpecName: "service-ca") pod "bba28cab-660e-43f1-abf1-471e2cc17889" (UID: "bba28cab-660e-43f1-abf1-471e2cc17889"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:09:52.104922 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104806 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bba28cab-660e-43f1-abf1-471e2cc17889" (UID: "bba28cab-660e-43f1-abf1-471e2cc17889"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:09:52.104922 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.104894 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-console-config" (OuterVolumeSpecName: "console-config") pod "bba28cab-660e-43f1-abf1-471e2cc17889" (UID: "bba28cab-660e-43f1-abf1-471e2cc17889"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:09:52.106516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.106490 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba28cab-660e-43f1-abf1-471e2cc17889-kube-api-access-przlx" (OuterVolumeSpecName: "kube-api-access-przlx") pod "bba28cab-660e-43f1-abf1-471e2cc17889" (UID: "bba28cab-660e-43f1-abf1-471e2cc17889"). InnerVolumeSpecName "kube-api-access-przlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:09:52.106516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.106501 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bba28cab-660e-43f1-abf1-471e2cc17889" (UID: "bba28cab-660e-43f1-abf1-471e2cc17889"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:09:52.106682 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.106641 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bba28cab-660e-43f1-abf1-471e2cc17889" (UID: "bba28cab-660e-43f1-abf1-471e2cc17889"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:09:52.205877 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.205843 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-przlx\" (UniqueName: \"kubernetes.io/projected/bba28cab-660e-43f1-abf1-471e2cc17889-kube-api-access-przlx\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:09:52.205877 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.205873 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-oauth-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:09:52.205877 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.205882 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bba28cab-660e-43f1-abf1-471e2cc17889-console-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:09:52.206077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.205891 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:09:52.206077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.205900 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-oauth-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:09:52.206077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.205907 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bba28cab-660e-43f1-abf1-471e2cc17889-console-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:09:52.964020 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.963994 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db4778974-llvgp" Aug 04 09:09:52.964463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.964018 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db4778974-llvgp" event={"ID":"bba28cab-660e-43f1-abf1-471e2cc17889","Type":"ContainerDied","Data":"667e85a89617646c78fd3885adfdc0c5b5c7c0c2ad2ff65f0de005e849058680"} Aug 04 09:09:52.964463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.964075 2562 scope.go:117] "RemoveContainer" containerID="cbbbe7b033b8aedb1bd16b35f95c599e4978b520f96a9b3faf19fccdf4f124cb" Aug 04 09:09:52.989472 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.989435 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db4778974-llvgp"] Aug 04 09:09:52.998474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:52.998453 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db4778974-llvgp"] Aug 04 09:09:53.415341 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:09:53.415310 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bba28cab-660e-43f1-abf1-471e2cc17889" path="/var/lib/kubelet/pods/bba28cab-660e-43f1-abf1-471e2cc17889/volumes" Aug 04 09:10:01.081863 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.081799 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fcfd957b5-l9xv8" podUID="f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" containerName="console" containerID="cri-o://77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83" gracePeriod=15 Aug 04 09:10:01.324176 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.324153 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:10:01.379341 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379270 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-service-ca\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379341 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379329 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwkns\" (UniqueName: \"kubernetes.io/projected/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-kube-api-access-gwkns\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379355 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-trusted-ca-bundle\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379371 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-oauth-serving-cert\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379401 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-config\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379419 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-serving-cert\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379735 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379569 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-oauth-config\") pod \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\" (UID: \"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce\") " Aug 04 09:10:01.379735 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379699 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-service-ca" (OuterVolumeSpecName: "service-ca") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:01.379869 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379836 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-config" (OuterVolumeSpecName: "console-config") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:01.379923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379855 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:01.379923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379873 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:01.379923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.379877 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:01.381521 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.381497 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:10:01.381623 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.381539 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:10:01.381623 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.381578 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-kube-api-access-gwkns" (OuterVolumeSpecName: "kube-api-access-gwkns") pod "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" (UID: "f656a6c0-67ce-4d3e-ae73-a0ecaac83cce"). InnerVolumeSpecName "kube-api-access-gwkns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:10:01.480800 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.480769 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwkns\" (UniqueName: \"kubernetes.io/projected/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-kube-api-access-gwkns\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:01.480800 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.480794 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-trusted-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:01.480800 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.480804 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-oauth-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:01.480986 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.480813 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:01.480986 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.480821 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:01.480986 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:01.480830 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce-console-oauth-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:02.008794 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.008763 2562 generic.go:356] "Generic (PLEG): container finished" podID="f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" containerID="77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83" exitCode=0 Aug 04 09:10:02.008967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.008808 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfd957b5-l9xv8" Aug 04 09:10:02.008967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.008812 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfd957b5-l9xv8" event={"ID":"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce","Type":"ContainerDied","Data":"77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83"} Aug 04 09:10:02.008967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.008854 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfd957b5-l9xv8" event={"ID":"f656a6c0-67ce-4d3e-ae73-a0ecaac83cce","Type":"ContainerDied","Data":"3525f41f6e2dffc732e10cd295457c69b9f9feadf9e6569a29dd281e4c52f7ef"} Aug 04 09:10:02.008967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.008874 2562 scope.go:117] "RemoveContainer" containerID="77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83" Aug 04 09:10:02.016815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.016798 2562 scope.go:117] "RemoveContainer" containerID="77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83" Aug 04 09:10:02.017053 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:10:02.017030 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83\": container with ID starting with 77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83 not found: ID does not exist" containerID="77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83" Aug 04 09:10:02.017141 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.017058 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83"} err="failed to get container status \"77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83\": rpc error: code = NotFound desc = could not find container \"77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83\": container with ID starting with 77af080264b1ee1d288b43f3938bf1426dfb3d855e2e5bf9422ca50c856d5a83 not found: ID does not exist" Aug 04 09:10:02.026593 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.026568 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcfd957b5-l9xv8"] Aug 04 09:10:02.030760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:02.030739 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fcfd957b5-l9xv8"] Aug 04 09:10:03.415264 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:03.415220 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" path="/var/lib/kubelet/pods/f656a6c0-67ce-4d3e-ae73-a0ecaac83cce/volumes" Aug 04 09:10:12.008528 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.008468 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55c77485f5-rbgdv" podUID="8af6371e-d13f-45cc-8f8a-2963b1f98aae" containerName="console" containerID="cri-o://622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92" gracePeriod=15 Aug 04 09:10:12.243234 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.243210 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:10:12.370305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370224 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbvft\" (UniqueName: \"kubernetes.io/projected/8af6371e-d13f-45cc-8f8a-2963b1f98aae-kube-api-access-wbvft\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370283 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-oauth-serving-cert\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-service-ca\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370377 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-trusted-ca-bundle\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370414 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-oauth-config\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370451 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-serving-cert\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370720 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-config\") pod \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\" (UID: \"8af6371e-d13f-45cc-8f8a-2963b1f98aae\") " Aug 04 09:10:12.370760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370747 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:12.371014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370949 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-service-ca" (OuterVolumeSpecName: "service-ca") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:12.371014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.370955 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:12.371306 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.371247 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-config" (OuterVolumeSpecName: "console-config") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:10:12.371445 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.371377 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-oauth-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:12.371445 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.371401 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:12.371445 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.371415 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-trusted-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:12.372730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.372703 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af6371e-d13f-45cc-8f8a-2963b1f98aae-kube-api-access-wbvft" (OuterVolumeSpecName: "kube-api-access-wbvft") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "kube-api-access-wbvft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:10:12.373068 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.373049 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:10:12.373153 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.373123 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8af6371e-d13f-45cc-8f8a-2963b1f98aae" (UID: "8af6371e-d13f-45cc-8f8a-2963b1f98aae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:10:12.472468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.472439 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-oauth-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:12.472468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.472463 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:12.472636 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.472478 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af6371e-d13f-45cc-8f8a-2963b1f98aae-console-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:12.472636 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:12.472487 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbvft\" (UniqueName: \"kubernetes.io/projected/8af6371e-d13f-45cc-8f8a-2963b1f98aae-kube-api-access-wbvft\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:10:13.053351 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.053312 2562 generic.go:356] "Generic (PLEG): container finished" podID="8af6371e-d13f-45cc-8f8a-2963b1f98aae" containerID="622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92" exitCode=0 Aug 04 09:10:13.053776 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.053370 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c77485f5-rbgdv" Aug 04 09:10:13.053776 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.053390 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c77485f5-rbgdv" event={"ID":"8af6371e-d13f-45cc-8f8a-2963b1f98aae","Type":"ContainerDied","Data":"622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92"} Aug 04 09:10:13.053776 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.053463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c77485f5-rbgdv" event={"ID":"8af6371e-d13f-45cc-8f8a-2963b1f98aae","Type":"ContainerDied","Data":"99d654e637f79271436f721b3bd7a43c144c513ec088872708cbc0e59fe188f5"} Aug 04 09:10:13.053776 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.053484 2562 scope.go:117] "RemoveContainer" containerID="622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92" Aug 04 09:10:13.061778 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.061754 2562 scope.go:117] "RemoveContainer" containerID="622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92" Aug 04 09:10:13.062031 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:10:13.062012 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92\": container with ID starting with 622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92 not found: ID does not exist" containerID="622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92" Aug 04 09:10:13.062096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.062041 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92"} err="failed to get container status \"622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92\": rpc error: code = NotFound desc = could not find container \"622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92\": container with ID starting with 622aeae240d905d6e0a424736589dc516a294ff6517bd303053ba8816539dc92 not found: ID does not exist" Aug 04 09:10:13.074021 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.073994 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c77485f5-rbgdv"] Aug 04 09:10:13.079141 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.079120 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55c77485f5-rbgdv"] Aug 04 09:10:13.415687 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.415655 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8af6371e-d13f-45cc-8f8a-2963b1f98aae" path="/var/lib/kubelet/pods/8af6371e-d13f-45cc-8f8a-2963b1f98aae/volumes" Aug 04 09:10:13.629103 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.629074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:13.691821 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:13.691755 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:43.157693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:43.157655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:43.233424 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:43.233391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:46.865021 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.864987 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8474dd7d9c-7x6j9"] Aug 04 09:10:46.865396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865300 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" containerName="console" Aug 04 09:10:46.865396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865313 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" containerName="console" Aug 04 09:10:46.865396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865334 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af6371e-d13f-45cc-8f8a-2963b1f98aae" containerName="console" Aug 04 09:10:46.865396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865340 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af6371e-d13f-45cc-8f8a-2963b1f98aae" containerName="console" Aug 04 09:10:46.865396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865352 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bba28cab-660e-43f1-abf1-471e2cc17889" containerName="console" Aug 04 09:10:46.865396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865357 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bba28cab-660e-43f1-abf1-471e2cc17889" containerName="console" Aug 04 09:10:46.865583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865402 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bba28cab-660e-43f1-abf1-471e2cc17889" containerName="console" Aug 04 09:10:46.865583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865411 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f656a6c0-67ce-4d3e-ae73-a0ecaac83cce" containerName="console" Aug 04 09:10:46.865583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.865419 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af6371e-d13f-45cc-8f8a-2963b1f98aae" containerName="console" Aug 04 09:10:46.869546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.869524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.896208 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.896182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8474dd7d9c-7x6j9"] Aug 04 09:10:46.940901 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.940871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:46.950080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-serving-cert\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.950158 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-trusted-ca-bundle\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.950158 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950119 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-config\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.950234 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ktm9\" (UniqueName: \"kubernetes.io/projected/70506efe-201b-4fa1-a1e7-8aa06e72635e-kube-api-access-2ktm9\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.950234 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950200 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-oauth-config\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.950314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-service-ca\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:46.950347 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:46.950323 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-oauth-serving-cert\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-serving-cert\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-trusted-ca-bundle\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-config\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ktm9\" (UniqueName: \"kubernetes.io/projected/70506efe-201b-4fa1-a1e7-8aa06e72635e-kube-api-access-2ktm9\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-oauth-config\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051829 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-service-ca\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.051829 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.051653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-oauth-serving-cert\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.052386 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.052359 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-oauth-serving-cert\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.052510 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.052442 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-service-ca\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.052510 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.052494 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-config\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.052716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.052699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-trusted-ca-bundle\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.054171 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.054151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-oauth-config\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.054377 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.054361 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-serving-cert\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.059823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.059799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:47.061612 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.061584 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ktm9\" (UniqueName: \"kubernetes.io/projected/70506efe-201b-4fa1-a1e7-8aa06e72635e-kube-api-access-2ktm9\") pod \"console-8474dd7d9c-7x6j9\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.178330 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.178291 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:47.303792 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:47.303756 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8474dd7d9c-7x6j9"] Aug 04 09:10:47.305852 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:10:47.305829 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70506efe_201b_4fa1_a1e7_8aa06e72635e.slice/crio-78660bc1de2c4de54624db16f721bb3efed506ce1ba19145cf3e87b4adf98bdc WatchSource:0}: Error finding container 78660bc1de2c4de54624db16f721bb3efed506ce1ba19145cf3e87b4adf98bdc: Status 404 returned error can't find the container with id 78660bc1de2c4de54624db16f721bb3efed506ce1ba19145cf3e87b4adf98bdc Aug 04 09:10:48.181493 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:48.181455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474dd7d9c-7x6j9" event={"ID":"70506efe-201b-4fa1-a1e7-8aa06e72635e","Type":"ContainerStarted","Data":"f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d"} Aug 04 09:10:48.181974 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:48.181511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474dd7d9c-7x6j9" event={"ID":"70506efe-201b-4fa1-a1e7-8aa06e72635e","Type":"ContainerStarted","Data":"78660bc1de2c4de54624db16f721bb3efed506ce1ba19145cf3e87b4adf98bdc"} Aug 04 09:10:48.202546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:48.202500 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8474dd7d9c-7x6j9" podStartSLOduration=2.202484607 podStartE2EDuration="2.202484607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:10:48.200868089 +0000 UTC m=+165.333077631" watchObservedRunningTime="2026-08-04 09:10:48.202484607 +0000 UTC m=+165.334694139" Aug 04 09:10:55.243757 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:55.243671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:55.302666 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:55.302634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:57.179200 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.179161 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:57.179694 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.179232 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:57.183905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.183881 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:57.219640 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.219614 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:10:57.272813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.272785 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79bf8b8887-cc5vl"] Aug 04 09:10:57.394822 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.394786 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:57.495931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.495858 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:57.571544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:57.571509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:58.636019 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:58.635987 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:58.735720 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:58.735688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:10:58.797664 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:10:58.797635 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:11:09.307897 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:09.307863 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:11:10.833143 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:10.833110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:11:17.054600 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:17.054564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:11:22.291683 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.291619 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79bf8b8887-cc5vl" podUID="6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" containerName="console" containerID="cri-o://ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2" gracePeriod=15 Aug 04 09:11:22.521664 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.521635 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:11:22.655453 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655427 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-trusted-ca-bundle\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655471 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-service-ca\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655542 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-oauth-config\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655595 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtkbz\" (UniqueName: \"kubernetes.io/projected/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-kube-api-access-vtkbz\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655617 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-serving-cert\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655846 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655694 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-oauth-serving-cert\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655846 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655718 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-config\") pod \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\" (UID: \"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5\") " Aug 04 09:11:22.655931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.655876 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:11:22.656045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.656021 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-trusted-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:22.656045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.656017 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:11:22.656215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.656125 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:11:22.656215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.656132 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-config" (OuterVolumeSpecName: "console-config") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:11:22.657738 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.657710 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:11:22.657822 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.657764 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:11:22.657822 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.657772 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-kube-api-access-vtkbz" (OuterVolumeSpecName: "kube-api-access-vtkbz") pod "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" (UID: "6ceb4474-3ca0-434f-8ee8-d8793f5f08d5"). InnerVolumeSpecName "kube-api-access-vtkbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:11:22.757364 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.757326 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-oauth-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:22.757364 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.757355 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtkbz\" (UniqueName: \"kubernetes.io/projected/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-kube-api-access-vtkbz\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:22.757364 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.757364 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:22.757364 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.757372 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-oauth-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:22.757630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.757383 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-console-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:22.757630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:22.757392 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:11:23.310281 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.310232 2562 generic.go:356] "Generic (PLEG): container finished" podID="6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" containerID="ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2" exitCode=0 Aug 04 09:11:23.310763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.310312 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf8b8887-cc5vl" Aug 04 09:11:23.310763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.310322 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf8b8887-cc5vl" event={"ID":"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5","Type":"ContainerDied","Data":"ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2"} Aug 04 09:11:23.310763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.310395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf8b8887-cc5vl" event={"ID":"6ceb4474-3ca0-434f-8ee8-d8793f5f08d5","Type":"ContainerDied","Data":"f955ccd3af1302ce32d3e78aafd07b412eb61c5efce66e38d099d810b2f1980e"} Aug 04 09:11:23.310763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.310416 2562 scope.go:117] "RemoveContainer" containerID="ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2" Aug 04 09:11:23.318439 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.318423 2562 scope.go:117] "RemoveContainer" containerID="ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2" Aug 04 09:11:23.318667 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:11:23.318650 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2\": container with ID starting with ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2 not found: ID does not exist" containerID="ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2" Aug 04 09:11:23.318733 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.318672 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2"} err="failed to get container status \"ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2\": rpc error: code = NotFound desc = could not find container \"ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2\": container with ID starting with ec167fe0bb566c07d7663caa0ab4f76d49a93134a93c8d9d61bdb429f6b708c2 not found: ID does not exist" Aug 04 09:11:23.342337 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.342315 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79bf8b8887-cc5vl"] Aug 04 09:11:23.358196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.358174 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79bf8b8887-cc5vl"] Aug 04 09:11:23.415366 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:11:23.415341 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" path="/var/lib/kubelet/pods/6ceb4474-3ca0-434f-8ee8-d8793f5f08d5/volumes" Aug 04 09:12:12.567197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:12.567161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:12.665304 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:12.665278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:13.324511 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.324479 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb"] Aug 04 09:12:13.324837 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.324825 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" containerName="console" Aug 04 09:12:13.324890 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.324839 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" containerName="console" Aug 04 09:12:13.324926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.324898 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ceb4474-3ca0-434f-8ee8-d8793f5f08d5" containerName="console" Aug 04 09:12:13.328321 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.328305 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.330932 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.330906 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:12:13.331066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.330962 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:12:13.331066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.330984 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v49jl\"" Aug 04 09:12:13.338957 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.338936 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb"] Aug 04 09:12:13.455082 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.455047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.455082 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.455086 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.455315 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.455193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmdh6\" (UniqueName: \"kubernetes.io/projected/569ec5f6-5e93-450e-8edb-21c4b1df00a6-kube-api-access-fmdh6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.555624 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.555588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmdh6\" (UniqueName: \"kubernetes.io/projected/569ec5f6-5e93-450e-8edb-21c4b1df00a6-kube-api-access-fmdh6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.555774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.555681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.555774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.555712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.556095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.556074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.556129 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.556102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.563913 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.563885 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmdh6\" (UniqueName: \"kubernetes.io/projected/569ec5f6-5e93-450e-8edb-21c4b1df00a6-kube-api-access-fmdh6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.637936 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.637846 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:13.767756 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:13.767727 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb"] Aug 04 09:12:13.769659 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:13.769636 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569ec5f6_5e93_450e_8edb_21c4b1df00a6.slice/crio-060bd590129572194b6bde1039b9764ef9db420c6d7756fdd9f83e66378afb8b WatchSource:0}: Error finding container 060bd590129572194b6bde1039b9764ef9db420c6d7756fdd9f83e66378afb8b: Status 404 returned error can't find the container with id 060bd590129572194b6bde1039b9764ef9db420c6d7756fdd9f83e66378afb8b Aug 04 09:12:14.497394 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:14.497361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" event={"ID":"569ec5f6-5e93-450e-8edb-21c4b1df00a6","Type":"ContainerStarted","Data":"060bd590129572194b6bde1039b9764ef9db420c6d7756fdd9f83e66378afb8b"} Aug 04 09:12:23.533469 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:23.533431 2562 generic.go:356] "Generic (PLEG): container finished" podID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerID="acd0227e58f5cb32c8dd57f6a663ea651e56b022c6e26ca249c76b6848198a95" exitCode=0 Aug 04 09:12:23.533945 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:23.533485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" event={"ID":"569ec5f6-5e93-450e-8edb-21c4b1df00a6","Type":"ContainerDied","Data":"acd0227e58f5cb32c8dd57f6a663ea651e56b022c6e26ca249c76b6848198a95"} Aug 04 09:12:26.547928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:26.547890 2562 generic.go:356] "Generic (PLEG): container finished" podID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerID="2bcbb3c32a7769d83d24a8114d7afc00ffe522a7224a35d3fef4d46862b95a89" exitCode=0 Aug 04 09:12:26.548416 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:26.547942 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" event={"ID":"569ec5f6-5e93-450e-8edb-21c4b1df00a6","Type":"ContainerDied","Data":"2bcbb3c32a7769d83d24a8114d7afc00ffe522a7224a35d3fef4d46862b95a89"} Aug 04 09:12:31.125962 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:31.125919 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:31.183905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:31.183867 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:34.587666 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:34.587577 2562 generic.go:356] "Generic (PLEG): container finished" podID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerID="31989b4e1c0fc38f74627f1e50654582a0a46832c436ce1abd0c4cd4be2b28dc" exitCode=0 Aug 04 09:12:34.587666 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:34.587651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" event={"ID":"569ec5f6-5e93-450e-8edb-21c4b1df00a6","Type":"ContainerDied","Data":"31989b4e1c0fc38f74627f1e50654582a0a46832c436ce1abd0c4cd4be2b28dc"} Aug 04 09:12:35.734432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.734402 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:35.841461 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.841431 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmdh6\" (UniqueName: \"kubernetes.io/projected/569ec5f6-5e93-450e-8edb-21c4b1df00a6-kube-api-access-fmdh6\") pod \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " Aug 04 09:12:35.841621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.841486 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-bundle\") pod \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " Aug 04 09:12:35.841621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.841528 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-util\") pod \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\" (UID: \"569ec5f6-5e93-450e-8edb-21c4b1df00a6\") " Aug 04 09:12:35.842184 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.842142 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-bundle" (OuterVolumeSpecName: "bundle") pod "569ec5f6-5e93-450e-8edb-21c4b1df00a6" (UID: "569ec5f6-5e93-450e-8edb-21c4b1df00a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:12:35.843631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.843603 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/569ec5f6-5e93-450e-8edb-21c4b1df00a6-kube-api-access-fmdh6" (OuterVolumeSpecName: "kube-api-access-fmdh6") pod "569ec5f6-5e93-450e-8edb-21c4b1df00a6" (UID: "569ec5f6-5e93-450e-8edb-21c4b1df00a6"). InnerVolumeSpecName "kube-api-access-fmdh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:12:35.846038 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.846010 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-util" (OuterVolumeSpecName: "util") pod "569ec5f6-5e93-450e-8edb-21c4b1df00a6" (UID: "569ec5f6-5e93-450e-8edb-21c4b1df00a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:12:35.942841 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.942812 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmdh6\" (UniqueName: \"kubernetes.io/projected/569ec5f6-5e93-450e-8edb-21c4b1df00a6-kube-api-access-fmdh6\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:12:35.942841 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.942836 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:12:35.942841 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:35.942845 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/569ec5f6-5e93-450e-8edb-21c4b1df00a6-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:12:36.598287 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:36.598225 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" event={"ID":"569ec5f6-5e93-450e-8edb-21c4b1df00a6","Type":"ContainerDied","Data":"060bd590129572194b6bde1039b9764ef9db420c6d7756fdd9f83e66378afb8b"} Aug 04 09:12:36.598287 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:36.598271 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qbjcb" Aug 04 09:12:36.598496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:36.598305 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="060bd590129572194b6bde1039b9764ef9db420c6d7756fdd9f83e66378afb8b" Aug 04 09:12:39.394470 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:39.394444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:41.141892 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.141857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:41.360894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.360857 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567"] Aug 04 09:12:41.361222 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361204 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="util" Aug 04 09:12:41.361331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361224 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="util" Aug 04 09:12:41.361331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361277 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="extract" Aug 04 09:12:41.361331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361287 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="extract" Aug 04 09:12:41.361331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361307 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="pull" Aug 04 09:12:41.361331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361315 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="pull" Aug 04 09:12:41.361574 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.361384 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="569ec5f6-5e93-450e-8edb-21c4b1df00a6" containerName="extract" Aug 04 09:12:41.364468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.364448 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.366984 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.366956 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-wm6gh\"" Aug 04 09:12:41.367112 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.366998 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Aug 04 09:12:41.367112 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.367001 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:12:41.379560 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.379536 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567"] Aug 04 09:12:41.488081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.488003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1e194ca-825f-4aa1-acb8-6103352e3ad6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wb567\" (UID: \"d1e194ca-825f-4aa1-acb8-6103352e3ad6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.488081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.488057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48b5w\" (UniqueName: \"kubernetes.io/projected/d1e194ca-825f-4aa1-acb8-6103352e3ad6-kube-api-access-48b5w\") pod \"cert-manager-operator-controller-manager-55c444bf86-wb567\" (UID: \"d1e194ca-825f-4aa1-acb8-6103352e3ad6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.589506 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.589476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1e194ca-825f-4aa1-acb8-6103352e3ad6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wb567\" (UID: \"d1e194ca-825f-4aa1-acb8-6103352e3ad6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.589664 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.589531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48b5w\" (UniqueName: \"kubernetes.io/projected/d1e194ca-825f-4aa1-acb8-6103352e3ad6-kube-api-access-48b5w\") pod \"cert-manager-operator-controller-manager-55c444bf86-wb567\" (UID: \"d1e194ca-825f-4aa1-acb8-6103352e3ad6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.589859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.589842 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1e194ca-825f-4aa1-acb8-6103352e3ad6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wb567\" (UID: \"d1e194ca-825f-4aa1-acb8-6103352e3ad6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.600366 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.600336 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48b5w\" (UniqueName: \"kubernetes.io/projected/d1e194ca-825f-4aa1-acb8-6103352e3ad6-kube-api-access-48b5w\") pod \"cert-manager-operator-controller-manager-55c444bf86-wb567\" (UID: \"d1e194ca-825f-4aa1-acb8-6103352e3ad6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.673633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.673595 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" Aug 04 09:12:41.812288 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:41.812246 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567"] Aug 04 09:12:41.815494 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:41.815462 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e194ca_825f_4aa1_acb8_6103352e3ad6.slice/crio-ccb9ee3223152121be6aefabff484f52c1e6b576c3bfda0647d2dc2c4a4af17f WatchSource:0}: Error finding container ccb9ee3223152121be6aefabff484f52c1e6b576c3bfda0647d2dc2c4a4af17f: Status 404 returned error can't find the container with id ccb9ee3223152121be6aefabff484f52c1e6b576c3bfda0647d2dc2c4a4af17f Aug 04 09:12:42.623595 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:42.623558 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" event={"ID":"d1e194ca-825f-4aa1-acb8-6103352e3ad6","Type":"ContainerStarted","Data":"ccb9ee3223152121be6aefabff484f52c1e6b576c3bfda0647d2dc2c4a4af17f"} Aug 04 09:12:43.013134 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:43.013094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:43.113025 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:43.112991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:44.635573 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:44.635541 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" event={"ID":"d1e194ca-825f-4aa1-acb8-6103352e3ad6","Type":"ContainerStarted","Data":"9848b75459d3fe38a777fd654198e790bdd91bf961141125815bfd08e573df72"} Aug 04 09:12:44.670722 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:44.670669 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wb567" podStartSLOduration=1.913984715 podStartE2EDuration="3.670651021s" totalImagesPullingTime="1.756666306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:12:44.669006333 +0000 UTC m=+281.801215877" watchObservedRunningTime="2026-08-04 09:12:44.670651021 +0000 UTC m=+281.802860553" Aug 04 09:12:45.040480 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.040445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:45.149137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.149104 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:45.277625 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.277597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:45.393825 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.393790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:45.423671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.423645 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk"] Aug 04 09:12:45.425803 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.425788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.428915 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.428890 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:12:45.428915 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.428905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:12:45.429812 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.429793 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v49jl\"" Aug 04 09:12:45.449861 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.449840 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk"] Aug 04 09:12:45.528340 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.528309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6292f\" (UniqueName: \"kubernetes.io/projected/78f20afb-fbe5-4505-9963-75458818adf4-kube-api-access-6292f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.528484 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.528368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.528484 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.528430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.629083 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.629047 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.629239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.629095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.629239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.629138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6292f\" (UniqueName: \"kubernetes.io/projected/78f20afb-fbe5-4505-9963-75458818adf4-kube-api-access-6292f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.629464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.629442 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.629518 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.629497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.640275 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.640229 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6292f\" (UniqueName: \"kubernetes.io/projected/78f20afb-fbe5-4505-9963-75458818adf4-kube-api-access-6292f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.734606 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.734525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:45.857849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:45.857827 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk"] Aug 04 09:12:45.860279 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:45.860220 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f20afb_fbe5_4505_9963_75458818adf4.slice/crio-75b851a87ea93f9ecc92d650393d20f35d81dd9ac32b4d3c1dee8ec428ab1eb0 WatchSource:0}: Error finding container 75b851a87ea93f9ecc92d650393d20f35d81dd9ac32b4d3c1dee8ec428ab1eb0: Status 404 returned error can't find the container with id 75b851a87ea93f9ecc92d650393d20f35d81dd9ac32b4d3c1dee8ec428ab1eb0 Aug 04 09:12:46.649971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:46.648526 2562 generic.go:356] "Generic (PLEG): container finished" podID="78f20afb-fbe5-4505-9963-75458818adf4" containerID="bbf9c5045543caa1f3c86465875a93b36771de775dc3c88ca45fe489eb68f9c6" exitCode=0 Aug 04 09:12:46.649971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:46.648677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" event={"ID":"78f20afb-fbe5-4505-9963-75458818adf4","Type":"ContainerDied","Data":"bbf9c5045543caa1f3c86465875a93b36771de775dc3c88ca45fe489eb68f9c6"} Aug 04 09:12:46.649971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:46.648756 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" event={"ID":"78f20afb-fbe5-4505-9963-75458818adf4","Type":"ContainerStarted","Data":"75b851a87ea93f9ecc92d650393d20f35d81dd9ac32b4d3c1dee8ec428ab1eb0"} Aug 04 09:12:47.648163 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:47.648123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:48.061828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.061748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:48.078874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.078846 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-c7dst"] Aug 04 09:12:48.081124 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.081103 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.083613 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.083566 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 09:12:48.083724 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.083667 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 09:12:48.084551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.084531 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z56d4\"" Aug 04 09:12:48.089796 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.089772 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-c7dst"] Aug 04 09:12:48.150581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.150551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkbb7\" (UniqueName: \"kubernetes.io/projected/f45ad3bb-d11c-4bab-bbd3-50b015837f19-kube-api-access-gkbb7\") pod \"cert-manager-cainjector-559cdd8c87-c7dst\" (UID: \"f45ad3bb-d11c-4bab-bbd3-50b015837f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.150731 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.150604 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f45ad3bb-d11c-4bab-bbd3-50b015837f19-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-c7dst\" (UID: \"f45ad3bb-d11c-4bab-bbd3-50b015837f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.251993 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.251957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f45ad3bb-d11c-4bab-bbd3-50b015837f19-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-c7dst\" (UID: \"f45ad3bb-d11c-4bab-bbd3-50b015837f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.252168 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.252060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkbb7\" (UniqueName: \"kubernetes.io/projected/f45ad3bb-d11c-4bab-bbd3-50b015837f19-kube-api-access-gkbb7\") pod \"cert-manager-cainjector-559cdd8c87-c7dst\" (UID: \"f45ad3bb-d11c-4bab-bbd3-50b015837f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.260670 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.260645 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f45ad3bb-d11c-4bab-bbd3-50b015837f19-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-c7dst\" (UID: \"f45ad3bb-d11c-4bab-bbd3-50b015837f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.260782 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.260654 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkbb7\" (UniqueName: \"kubernetes.io/projected/f45ad3bb-d11c-4bab-bbd3-50b015837f19-kube-api-access-gkbb7\") pod \"cert-manager-cainjector-559cdd8c87-c7dst\" (UID: \"f45ad3bb-d11c-4bab-bbd3-50b015837f19\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.399223 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.399192 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" Aug 04 09:12:48.946154 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:48.946081 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-c7dst"] Aug 04 09:12:48.948110 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:48.948088 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf45ad3bb_d11c_4bab_bbd3_50b015837f19.slice/crio-0aa7814f3b7259f0fff15ab6f23bb8f270e7905083471f99e95f52e6c84a0e08 WatchSource:0}: Error finding container 0aa7814f3b7259f0fff15ab6f23bb8f270e7905083471f99e95f52e6c84a0e08: Status 404 returned error can't find the container with id 0aa7814f3b7259f0fff15ab6f23bb8f270e7905083471f99e95f52e6c84a0e08 Aug 04 09:12:49.609771 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:49.609699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:49.665794 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:49.665762 2562 generic.go:356] "Generic (PLEG): container finished" podID="78f20afb-fbe5-4505-9963-75458818adf4" containerID="de2f23c3d9162a7d1d11bad9672ba6c34274a22416915ad331415752920add98" exitCode=0 Aug 04 09:12:49.665973 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:49.665852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" event={"ID":"78f20afb-fbe5-4505-9963-75458818adf4","Type":"ContainerDied","Data":"de2f23c3d9162a7d1d11bad9672ba6c34274a22416915ad331415752920add98"} Aug 04 09:12:49.667408 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:49.667382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" event={"ID":"f45ad3bb-d11c-4bab-bbd3-50b015837f19","Type":"ContainerStarted","Data":"0aa7814f3b7259f0fff15ab6f23bb8f270e7905083471f99e95f52e6c84a0e08"} Aug 04 09:12:49.675548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:49.675437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:49.837043 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:12:49.837012 2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f20afb_fbe5_4505_9963_75458818adf4.slice/crio-0aab522af9f2acf1b37a1b3565a1bd10ea9f57de0af5ef9e72e1b5a4d8d112dc.scope\": RecentStats: unable to find data in memory cache]" Aug 04 09:12:50.674671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:50.674629 2562 generic.go:356] "Generic (PLEG): container finished" podID="78f20afb-fbe5-4505-9963-75458818adf4" containerID="0aab522af9f2acf1b37a1b3565a1bd10ea9f57de0af5ef9e72e1b5a4d8d112dc" exitCode=0 Aug 04 09:12:50.675094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:50.674736 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" event={"ID":"78f20afb-fbe5-4505-9963-75458818adf4","Type":"ContainerDied","Data":"0aab522af9f2acf1b37a1b3565a1bd10ea9f57de0af5ef9e72e1b5a4d8d112dc"} Aug 04 09:12:51.680838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.680797 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" event={"ID":"f45ad3bb-d11c-4bab-bbd3-50b015837f19","Type":"ContainerStarted","Data":"ef31c3bd90b3fe6b6fd5c668573b17cd3268659bd3b5d0970d79ade0109e5e08"} Aug 04 09:12:51.702587 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.702512 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c7dst" podStartSLOduration=1.097957606 podStartE2EDuration="3.702489812s" totalImagesPullingTime="2.604532206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:12:51.700862497 +0000 UTC m=+288.833072029" watchObservedRunningTime="2026-08-04 09:12:51.702489812 +0000 UTC m=+288.834699345" Aug 04 09:12:51.737801 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.737767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:51.809458 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.809397 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:51.882201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.882171 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-bundle\") pod \"78f20afb-fbe5-4505-9963-75458818adf4\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " Aug 04 09:12:51.882374 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.882219 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6292f\" (UniqueName: \"kubernetes.io/projected/78f20afb-fbe5-4505-9963-75458818adf4-kube-api-access-6292f\") pod \"78f20afb-fbe5-4505-9963-75458818adf4\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " Aug 04 09:12:51.882374 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.882236 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-util\") pod \"78f20afb-fbe5-4505-9963-75458818adf4\" (UID: \"78f20afb-fbe5-4505-9963-75458818adf4\") " Aug 04 09:12:51.882546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.882517 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-bundle" (OuterVolumeSpecName: "bundle") pod "78f20afb-fbe5-4505-9963-75458818adf4" (UID: "78f20afb-fbe5-4505-9963-75458818adf4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:12:51.884362 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.884333 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f20afb-fbe5-4505-9963-75458818adf4-kube-api-access-6292f" (OuterVolumeSpecName: "kube-api-access-6292f") pod "78f20afb-fbe5-4505-9963-75458818adf4" (UID: "78f20afb-fbe5-4505-9963-75458818adf4"). InnerVolumeSpecName "kube-api-access-6292f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:12:51.886780 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.886755 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-util" (OuterVolumeSpecName: "util") pod "78f20afb-fbe5-4505-9963-75458818adf4" (UID: "78f20afb-fbe5-4505-9963-75458818adf4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:12:51.947068 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947037 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d55pt"] Aug 04 09:12:51.947369 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947356 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="pull" Aug 04 09:12:51.947417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947371 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="pull" Aug 04 09:12:51.947417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947385 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="util" Aug 04 09:12:51.947417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947390 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="util" Aug 04 09:12:51.947417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947405 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="extract" Aug 04 09:12:51.947417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947411 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="extract" Aug 04 09:12:51.947552 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.947460 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f20afb-fbe5-4505-9963-75458818adf4" containerName="extract" Aug 04 09:12:51.950499 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.950482 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:51.952846 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.952827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2l87r\"" Aug 04 09:12:51.961038 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.961017 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d55pt"] Aug 04 09:12:51.983538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.983513 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:12:51.983538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.983533 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6292f\" (UniqueName: \"kubernetes.io/projected/78f20afb-fbe5-4505-9963-75458818adf4-kube-api-access-6292f\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:12:51.983664 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:51.983543 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78f20afb-fbe5-4505-9963-75458818adf4-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:12:52.084491 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.084414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad985665-fb1a-443e-9fde-0d36d5f1191b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d55pt\" (UID: \"ad985665-fb1a-443e-9fde-0d36d5f1191b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.084491 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.084469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtxm\" (UniqueName: \"kubernetes.io/projected/ad985665-fb1a-443e-9fde-0d36d5f1191b-kube-api-access-pxtxm\") pod \"cert-manager-webhook-6486ffd997-d55pt\" (UID: \"ad985665-fb1a-443e-9fde-0d36d5f1191b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.185919 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.185885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad985665-fb1a-443e-9fde-0d36d5f1191b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d55pt\" (UID: \"ad985665-fb1a-443e-9fde-0d36d5f1191b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.186078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.185957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtxm\" (UniqueName: \"kubernetes.io/projected/ad985665-fb1a-443e-9fde-0d36d5f1191b-kube-api-access-pxtxm\") pod \"cert-manager-webhook-6486ffd997-d55pt\" (UID: \"ad985665-fb1a-443e-9fde-0d36d5f1191b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.194400 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.194374 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad985665-fb1a-443e-9fde-0d36d5f1191b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d55pt\" (UID: \"ad985665-fb1a-443e-9fde-0d36d5f1191b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.194502 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.194427 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtxm\" (UniqueName: \"kubernetes.io/projected/ad985665-fb1a-443e-9fde-0d36d5f1191b-kube-api-access-pxtxm\") pod \"cert-manager-webhook-6486ffd997-d55pt\" (UID: \"ad985665-fb1a-443e-9fde-0d36d5f1191b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.258958 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.258929 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.394884 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.394862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d55pt"] Aug 04 09:12:52.396749 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:52.396726 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad985665_fb1a_443e_9fde_0d36d5f1191b.slice/crio-6841964deb14febeed6d14985b178e8254c6627ffa793bd370a0adef55107f21 WatchSource:0}: Error finding container 6841964deb14febeed6d14985b178e8254c6627ffa793bd370a0adef55107f21: Status 404 returned error can't find the container with id 6841964deb14febeed6d14985b178e8254c6627ffa793bd370a0adef55107f21 Aug 04 09:12:52.686034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.686000 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" event={"ID":"ad985665-fb1a-443e-9fde-0d36d5f1191b","Type":"ContainerStarted","Data":"9e27ea8a83ca9cf21d04910317a7c54a5eccb92e704b39dd516638223aab8ce7"} Aug 04 09:12:52.686034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.686045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" event={"ID":"ad985665-fb1a-443e-9fde-0d36d5f1191b","Type":"ContainerStarted","Data":"6841964deb14febeed6d14985b178e8254c6627ffa793bd370a0adef55107f21"} Aug 04 09:12:52.686571 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.686165 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:52.687793 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.687777 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" Aug 04 09:12:52.687870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.687789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjsrqk" event={"ID":"78f20afb-fbe5-4505-9963-75458818adf4","Type":"ContainerDied","Data":"75b851a87ea93f9ecc92d650393d20f35d81dd9ac32b4d3c1dee8ec428ab1eb0"} Aug 04 09:12:52.687870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.687827 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75b851a87ea93f9ecc92d650393d20f35d81dd9ac32b4d3c1dee8ec428ab1eb0" Aug 04 09:12:52.702451 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:52.702415 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" podStartSLOduration=1.702401437 podStartE2EDuration="1.702401437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:12:52.701743803 +0000 UTC m=+289.833953339" watchObservedRunningTime="2026-08-04 09:12:52.702401437 +0000 UTC m=+289.834610969" Aug 04 09:12:55.239224 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.239181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:55.808516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.808478 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqzxn"] Aug 04 09:12:55.811566 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.811541 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:55.813930 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.813904 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4m2sr\"" Aug 04 09:12:55.821244 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.821221 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqzxn"] Aug 04 09:12:55.915900 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.915870 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf6zb\" (UniqueName: \"kubernetes.io/projected/fdac8923-a446-4831-b455-1b5900376190-kube-api-access-wf6zb\") pod \"cert-manager-55b4d954d5-nqzxn\" (UID: \"fdac8923-a446-4831-b455-1b5900376190\") " pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:55.916024 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:55.915946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdac8923-a446-4831-b455-1b5900376190-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqzxn\" (UID: \"fdac8923-a446-4831-b455-1b5900376190\") " pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:56.017232 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.017201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdac8923-a446-4831-b455-1b5900376190-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqzxn\" (UID: \"fdac8923-a446-4831-b455-1b5900376190\") " pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:56.017381 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.017241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf6zb\" (UniqueName: \"kubernetes.io/projected/fdac8923-a446-4831-b455-1b5900376190-kube-api-access-wf6zb\") pod \"cert-manager-55b4d954d5-nqzxn\" (UID: \"fdac8923-a446-4831-b455-1b5900376190\") " pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:56.025182 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.025149 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdac8923-a446-4831-b455-1b5900376190-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqzxn\" (UID: \"fdac8923-a446-4831-b455-1b5900376190\") " pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:56.025317 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.025215 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf6zb\" (UniqueName: \"kubernetes.io/projected/fdac8923-a446-4831-b455-1b5900376190-kube-api-access-wf6zb\") pod \"cert-manager-55b4d954d5-nqzxn\" (UID: \"fdac8923-a446-4831-b455-1b5900376190\") " pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:56.120633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.120557 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nqzxn" Aug 04 09:12:56.246463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.246432 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqzxn"] Aug 04 09:12:56.248849 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:56.248821 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdac8923_a446_4831_b455_1b5900376190.slice/crio-6a7446df4d619ae15825f96e03873f4ae80ae7f3ddfe8c66f80ce826378ae726 WatchSource:0}: Error finding container 6a7446df4d619ae15825f96e03873f4ae80ae7f3ddfe8c66f80ce826378ae726: Status 404 returned error can't find the container with id 6a7446df4d619ae15825f96e03873f4ae80ae7f3ddfe8c66f80ce826378ae726 Aug 04 09:12:56.708517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.708477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nqzxn" event={"ID":"fdac8923-a446-4831-b455-1b5900376190","Type":"ContainerStarted","Data":"71d9d21f2dacaa8f2706a66c05b2a78daad87d258bcde3262fc22f28f10b9d2a"} Aug 04 09:12:56.708517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.708520 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nqzxn" event={"ID":"fdac8923-a446-4831-b455-1b5900376190","Type":"ContainerStarted","Data":"6a7446df4d619ae15825f96e03873f4ae80ae7f3ddfe8c66f80ce826378ae726"} Aug 04 09:12:56.725235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.725187 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nqzxn" podStartSLOduration=1.725172782 podStartE2EDuration="1.725172782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:12:56.724194521 +0000 UTC m=+293.856404072" watchObservedRunningTime="2026-08-04 09:12:56.725172782 +0000 UTC m=+293.857382314" Aug 04 09:12:56.892416 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.892384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:56.992609 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:56.992524 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:12:58.369802 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.369770 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb"] Aug 04 09:12:58.372945 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.372922 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.375585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.375560 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zlm4z\"" Aug 04 09:12:58.375867 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.375849 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:12:58.376477 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.376463 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 09:12:58.387143 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.387124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb"] Aug 04 09:12:58.540380 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.540345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtjc5\" (UniqueName: \"kubernetes.io/projected/ba598665-37f3-4641-917d-0d71e304b3a6-kube-api-access-wtjc5\") pod \"openshift-lws-operator-fd8ccff4c-2z4nb\" (UID: \"ba598665-37f3-4641-917d-0d71e304b3a6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.540550 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.540388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba598665-37f3-4641-917d-0d71e304b3a6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2z4nb\" (UID: \"ba598665-37f3-4641-917d-0d71e304b3a6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.641094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.641010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjc5\" (UniqueName: \"kubernetes.io/projected/ba598665-37f3-4641-917d-0d71e304b3a6-kube-api-access-wtjc5\") pod \"openshift-lws-operator-fd8ccff4c-2z4nb\" (UID: \"ba598665-37f3-4641-917d-0d71e304b3a6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.641094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.641054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba598665-37f3-4641-917d-0d71e304b3a6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2z4nb\" (UID: \"ba598665-37f3-4641-917d-0d71e304b3a6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.641439 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.641423 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba598665-37f3-4641-917d-0d71e304b3a6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2z4nb\" (UID: \"ba598665-37f3-4641-917d-0d71e304b3a6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.649239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.649212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtjc5\" (UniqueName: \"kubernetes.io/projected/ba598665-37f3-4641-917d-0d71e304b3a6-kube-api-access-wtjc5\") pod \"openshift-lws-operator-fd8ccff4c-2z4nb\" (UID: \"ba598665-37f3-4641-917d-0d71e304b3a6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.681834 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.681810 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" Aug 04 09:12:58.695932 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.695913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d55pt" Aug 04 09:12:58.819777 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:58.819753 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb"] Aug 04 09:12:58.822523 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:12:58.822497 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba598665_37f3_4641_917d_0d71e304b3a6.slice/crio-58a8f994d139ba4561dc546e9518eda2f6cc3e4c1c6da4368f728121257c173e WatchSource:0}: Error finding container 58a8f994d139ba4561dc546e9518eda2f6cc3e4c1c6da4368f728121257c173e: Status 404 returned error can't find the container with id 58a8f994d139ba4561dc546e9518eda2f6cc3e4c1c6da4368f728121257c173e Aug 04 09:12:59.725056 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:12:59.725017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" event={"ID":"ba598665-37f3-4641-917d-0d71e304b3a6","Type":"ContainerStarted","Data":"58a8f994d139ba4561dc546e9518eda2f6cc3e4c1c6da4368f728121257c173e"} Aug 04 09:13:00.731152 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:00.731114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" event={"ID":"ba598665-37f3-4641-917d-0d71e304b3a6","Type":"ContainerStarted","Data":"7dade81bff1eeffa4bb73ddc0bc32b70908ec782ec490179978462976dbef23d"} Aug 04 09:13:00.757811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:00.753830 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2z4nb" podStartSLOduration=1.009087889 podStartE2EDuration="2.753810789s" totalImagesPullingTime="1.7447229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:00.749617485 +0000 UTC m=+297.881827030" watchObservedRunningTime="2026-08-04 09:13:00.753810789 +0000 UTC m=+297.886020324" Aug 04 09:13:01.513939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:01.513901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:01.644588 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:01.644552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:03.269689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.269661 2562 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 09:13:03.743063 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.743033 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph"] Aug 04 09:13:03.746135 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.746116 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.748857 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.748842 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v49jl\"" Aug 04 09:13:03.749121 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.749106 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:13:03.749839 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.749825 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:13:03.754757 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.754737 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:03.759311 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.759294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph"] Aug 04 09:13:03.888699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.888666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.888882 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.888742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddhwt\" (UniqueName: \"kubernetes.io/projected/166c8623-fd09-4fd3-a7fd-b6cd00806b16-kube-api-access-ddhwt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.888882 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.888768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.989160 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.989124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.989356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.989179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.989356 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.989289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddhwt\" (UniqueName: \"kubernetes.io/projected/166c8623-fd09-4fd3-a7fd-b6cd00806b16-kube-api-access-ddhwt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.989519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.989498 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.989611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.989542 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:03.998171 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:03.998112 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddhwt\" (UniqueName: \"kubernetes.io/projected/166c8623-fd09-4fd3-a7fd-b6cd00806b16-kube-api-access-ddhwt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:04.054896 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:04.054868 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:04.184599 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:04.184565 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph"] Aug 04 09:13:04.187716 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:13:04.187689 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166c8623_fd09_4fd3_a7fd_b6cd00806b16.slice/crio-29bb961ff90e986df55df4f617212d6cc2cd8a396e7312e0867f80585d07471a WatchSource:0}: Error finding container 29bb961ff90e986df55df4f617212d6cc2cd8a396e7312e0867f80585d07471a: Status 404 returned error can't find the container with id 29bb961ff90e986df55df4f617212d6cc2cd8a396e7312e0867f80585d07471a Aug 04 09:13:04.750650 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:04.750614 2562 generic.go:356] "Generic (PLEG): container finished" podID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerID="7694b822dede87f13fefadd91cc29338ab0c76328bf1648715ef9f04c6b62026" exitCode=0 Aug 04 09:13:04.751051 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:04.750703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" event={"ID":"166c8623-fd09-4fd3-a7fd-b6cd00806b16","Type":"ContainerDied","Data":"7694b822dede87f13fefadd91cc29338ab0c76328bf1648715ef9f04c6b62026"} Aug 04 09:13:04.751051 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:04.750763 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" event={"ID":"166c8623-fd09-4fd3-a7fd-b6cd00806b16","Type":"ContainerStarted","Data":"29bb961ff90e986df55df4f617212d6cc2cd8a396e7312e0867f80585d07471a"} Aug 04 09:13:04.751130 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:04.751107 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:13:06.760484 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:06.760449 2562 generic.go:356] "Generic (PLEG): container finished" podID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerID="b846d067222bc5bf277a543bae551b7b9848e750a6d33c75b745f5738630f924" exitCode=0 Aug 04 09:13:06.760885 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:06.760533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" event={"ID":"166c8623-fd09-4fd3-a7fd-b6cd00806b16","Type":"ContainerDied","Data":"b846d067222bc5bf277a543bae551b7b9848e750a6d33c75b745f5738630f924"} Aug 04 09:13:07.767507 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:07.767472 2562 generic.go:356] "Generic (PLEG): container finished" podID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerID="a5629ace84c8c5e10b34a0013c2e36078cb60e273f6341959c1f958f961bc9a0" exitCode=0 Aug 04 09:13:07.767865 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:07.767529 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" event={"ID":"166c8623-fd09-4fd3-a7fd-b6cd00806b16","Type":"ContainerDied","Data":"a5629ace84c8c5e10b34a0013c2e36078cb60e273f6341959c1f958f961bc9a0"} Aug 04 09:13:08.897306 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.897283 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:08.930828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.930791 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-bundle\") pod \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " Aug 04 09:13:08.930978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.930854 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-util\") pod \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " Aug 04 09:13:08.930978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.930906 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddhwt\" (UniqueName: \"kubernetes.io/projected/166c8623-fd09-4fd3-a7fd-b6cd00806b16-kube-api-access-ddhwt\") pod \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\" (UID: \"166c8623-fd09-4fd3-a7fd-b6cd00806b16\") " Aug 04 09:13:08.931547 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.931517 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-bundle" (OuterVolumeSpecName: "bundle") pod "166c8623-fd09-4fd3-a7fd-b6cd00806b16" (UID: "166c8623-fd09-4fd3-a7fd-b6cd00806b16"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:13:08.932910 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.932884 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/166c8623-fd09-4fd3-a7fd-b6cd00806b16-kube-api-access-ddhwt" (OuterVolumeSpecName: "kube-api-access-ddhwt") pod "166c8623-fd09-4fd3-a7fd-b6cd00806b16" (UID: "166c8623-fd09-4fd3-a7fd-b6cd00806b16"). InnerVolumeSpecName "kube-api-access-ddhwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:13:08.938970 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:08.938943 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-util" (OuterVolumeSpecName: "util") pod "166c8623-fd09-4fd3-a7fd-b6cd00806b16" (UID: "166c8623-fd09-4fd3-a7fd-b6cd00806b16"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:13:09.032463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:09.032397 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddhwt\" (UniqueName: \"kubernetes.io/projected/166c8623-fd09-4fd3-a7fd-b6cd00806b16-kube-api-access-ddhwt\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:13:09.032463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:09.032425 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:13:09.032463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:09.032434 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/166c8623-fd09-4fd3-a7fd-b6cd00806b16-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:13:09.778092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:09.778018 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" Aug 04 09:13:09.778092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:09.778041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5zvph" event={"ID":"166c8623-fd09-4fd3-a7fd-b6cd00806b16","Type":"ContainerDied","Data":"29bb961ff90e986df55df4f617212d6cc2cd8a396e7312e0867f80585d07471a"} Aug 04 09:13:09.778092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:09.778080 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29bb961ff90e986df55df4f617212d6cc2cd8a396e7312e0867f80585d07471a" Aug 04 09:13:14.238887 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:14.238834 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:19.414526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:19.414491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:19.535343 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:19.535309 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:20.463117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:20.463082 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:20.554711 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:20.554672 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:21.296085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.296052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:21.549923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.549845 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:21.647248 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.647216 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:21.681423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681394 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg"] Aug 04 09:13:21.681726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681714 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="extract" Aug 04 09:13:21.681774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681728 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="extract" Aug 04 09:13:21.681774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681738 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="pull" Aug 04 09:13:21.681774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681759 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="pull" Aug 04 09:13:21.681774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681769 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="util" Aug 04 09:13:21.681774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681775 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="util" Aug 04 09:13:21.681920 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.681841 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="166c8623-fd09-4fd3-a7fd-b6cd00806b16" containerName="extract" Aug 04 09:13:21.689218 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.689198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.692011 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.691988 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-52t6g\"" Aug 04 09:13:21.692220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.692028 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 09:13:21.692325 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.692311 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 09:13:21.692398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.692378 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 09:13:21.692655 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.692642 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 09:13:21.703543 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.703518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg"] Aug 04 09:13:21.738790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.738760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9182437b-86c5-4255-8e0c-fa5a01d8b05a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.738914 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.738801 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmjxl\" (UniqueName: \"kubernetes.io/projected/9182437b-86c5-4255-8e0c-fa5a01d8b05a-kube-api-access-kmjxl\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.738914 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.738821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9182437b-86c5-4255-8e0c-fa5a01d8b05a-manifests\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.738914 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.738876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9182437b-86c5-4255-8e0c-fa5a01d8b05a-tmp\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.739071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.738967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9182437b-86c5-4255-8e0c-fa5a01d8b05a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.839612 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.839533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9182437b-86c5-4255-8e0c-fa5a01d8b05a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.839612 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.839576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmjxl\" (UniqueName: \"kubernetes.io/projected/9182437b-86c5-4255-8e0c-fa5a01d8b05a-kube-api-access-kmjxl\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.839612 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.839598 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9182437b-86c5-4255-8e0c-fa5a01d8b05a-manifests\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.839866 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.839616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9182437b-86c5-4255-8e0c-fa5a01d8b05a-tmp\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.839866 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.839674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9182437b-86c5-4255-8e0c-fa5a01d8b05a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.840037 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.840012 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9182437b-86c5-4255-8e0c-fa5a01d8b05a-manifests\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.842017 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.841990 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9182437b-86c5-4255-8e0c-fa5a01d8b05a-tmp\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.842130 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.842069 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9182437b-86c5-4255-8e0c-fa5a01d8b05a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.842175 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.842133 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9182437b-86c5-4255-8e0c-fa5a01d8b05a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.852537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.852509 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmjxl\" (UniqueName: \"kubernetes.io/projected/9182437b-86c5-4255-8e0c-fa5a01d8b05a-kube-api-access-kmjxl\") pod \"opendatahub-operator-controller-manager-7c79dd6bb-bzzrg\" (UID: \"9182437b-86c5-4255-8e0c-fa5a01d8b05a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:21.943751 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.943718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:21.999828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:21.999795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:22.057474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.057429 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:22.133276 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.133177 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg"] Aug 04 09:13:22.136559 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:13:22.136528 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9182437b_86c5_4255_8e0c_fa5a01d8b05a.slice/crio-c2d1ab54ce98c7979dbb3186be5bedb184cf6e643f425b8559baa72cbf844bdc WatchSource:0}: Error finding container c2d1ab54ce98c7979dbb3186be5bedb184cf6e643f425b8559baa72cbf844bdc: Status 404 returned error can't find the container with id c2d1ab54ce98c7979dbb3186be5bedb184cf6e643f425b8559baa72cbf844bdc Aug 04 09:13:22.172592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.172561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:22.269713 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.269682 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:22.363386 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.363355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:22.462651 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.462618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:22.837236 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:22.837158 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" event={"ID":"9182437b-86c5-4255-8e0c-fa5a01d8b05a","Type":"ContainerStarted","Data":"c2d1ab54ce98c7979dbb3186be5bedb184cf6e643f425b8559baa72cbf844bdc"} Aug 04 09:13:24.085668 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:24.085630 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:24.202050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:24.202018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:25.856501 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:25.856466 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" event={"ID":"9182437b-86c5-4255-8e0c-fa5a01d8b05a","Type":"ContainerStarted","Data":"0ac5945220453a06c2151ef4139b8a2d468164a796a66cefa2a3cfe4f50d7793"} Aug 04 09:13:26.863309 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:26.863273 2562 generic.go:356] "Generic (PLEG): container finished" podID="9182437b-86c5-4255-8e0c-fa5a01d8b05a" containerID="0ac5945220453a06c2151ef4139b8a2d468164a796a66cefa2a3cfe4f50d7793" exitCode=0 Aug 04 09:13:26.863714 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:26.863333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" event={"ID":"9182437b-86c5-4255-8e0c-fa5a01d8b05a","Type":"ContainerDied","Data":"0ac5945220453a06c2151ef4139b8a2d468164a796a66cefa2a3cfe4f50d7793"} Aug 04 09:13:27.870447 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:27.870414 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" event={"ID":"9182437b-86c5-4255-8e0c-fa5a01d8b05a","Type":"ContainerStarted","Data":"5480ec39a84d346c402c2e3f31676d0140d104b04fcc69603c09fe54f6876189"} Aug 04 09:13:27.870920 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:27.870534 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:27.909051 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:27.908994 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" podStartSLOduration=2.529331816 podStartE2EDuration="6.908974113s" totalImagesPullingTime="3.353543969s" totalInitContainerRuntime="1.026098328s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:27.902239889 +0000 UTC m=+325.034449421" watchObservedRunningTime="2026-08-04 09:13:27.908974113 +0000 UTC m=+325.041183646" Aug 04 09:13:33.677119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:33.677088 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:33.773689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:33.773654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:38.877719 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:38.877688 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c79dd6bb-bzzrg" Aug 04 09:13:41.272883 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:41.272847 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:41.444002 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:41.443970 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:42.036953 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.036919 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z"] Aug 04 09:13:42.046307 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.046277 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.049517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.049490 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 09:13:42.049643 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.049490 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Aug 04 09:13:42.049697 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.049643 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 09:13:42.050233 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.050218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Aug 04 09:13:42.050452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.050440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j5xst\"" Aug 04 09:13:42.063651 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.063627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z"] Aug 04 09:13:42.109671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.109628 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:42.182520 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.182487 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-94szl"] Aug 04 09:13:42.186127 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.186107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.189130 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.189113 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-t7zd9\"" Aug 04 09:13:42.189479 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.189464 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Aug 04 09:13:42.206912 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.206887 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-94szl"] Aug 04 09:13:42.218488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.218459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-tls-certs\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.218586 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.218537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-tmp\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.218634 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.218611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmx9p\" (UniqueName: \"kubernetes.io/projected/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-kube-api-access-kmx9p\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.256424 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.256393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:42.319107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.319048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmx9p\" (UniqueName: \"kubernetes.io/projected/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-kube-api-access-kmx9p\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.319107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.319087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2f2546a-5a85-4f91-ad18-6d404200fd5c-cert\") pod \"odh-model-controller-7998c67dc4-94szl\" (UID: \"b2f2546a-5a85-4f91-ad18-6d404200fd5c\") " pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.319588 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.319115 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-tls-certs\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.319588 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.319190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-tmp\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.319588 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.319278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5qq2\" (UniqueName: \"kubernetes.io/projected/b2f2546a-5a85-4f91-ad18-6d404200fd5c-kube-api-access-d5qq2\") pod \"odh-model-controller-7998c67dc4-94szl\" (UID: \"b2f2546a-5a85-4f91-ad18-6d404200fd5c\") " pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.321403 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.321371 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-tmp\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.321593 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.321575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-tls-certs\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.329339 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.329318 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmx9p\" (UniqueName: \"kubernetes.io/projected/3ebe8684-4d19-4144-96d7-a66c6fdb78e7-kube-api-access-kmx9p\") pod \"kube-auth-proxy-6bcc56ccff-xnn4z\" (UID: \"3ebe8684-4d19-4144-96d7-a66c6fdb78e7\") " pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.356103 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.356080 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" Aug 04 09:13:42.386199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.386166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:42.420480 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.420448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5qq2\" (UniqueName: \"kubernetes.io/projected/b2f2546a-5a85-4f91-ad18-6d404200fd5c-kube-api-access-d5qq2\") pod \"odh-model-controller-7998c67dc4-94szl\" (UID: \"b2f2546a-5a85-4f91-ad18-6d404200fd5c\") " pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.420653 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.420498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2f2546a-5a85-4f91-ad18-6d404200fd5c-cert\") pod \"odh-model-controller-7998c67dc4-94szl\" (UID: \"b2f2546a-5a85-4f91-ad18-6d404200fd5c\") " pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.424102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.424083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2f2546a-5a85-4f91-ad18-6d404200fd5c-cert\") pod \"odh-model-controller-7998c67dc4-94szl\" (UID: \"b2f2546a-5a85-4f91-ad18-6d404200fd5c\") " pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.430583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.430562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5qq2\" (UniqueName: \"kubernetes.io/projected/b2f2546a-5a85-4f91-ad18-6d404200fd5c-kube-api-access-d5qq2\") pod \"odh-model-controller-7998c67dc4-94szl\" (UID: \"b2f2546a-5a85-4f91-ad18-6d404200fd5c\") " pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.484708 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.484685 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z"] Aug 04 09:13:42.486887 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:13:42.486858 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebe8684_4d19_4144_96d7_a66c6fdb78e7.slice/crio-d8036780eb5fb9f416f574838df7cd9e438603020fe6f1d8ab9072f17ec40e68 WatchSource:0}: Error finding container d8036780eb5fb9f416f574838df7cd9e438603020fe6f1d8ab9072f17ec40e68: Status 404 returned error can't find the container with id d8036780eb5fb9f416f574838df7cd9e438603020fe6f1d8ab9072f17ec40e68 Aug 04 09:13:42.487006 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.486933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:42.495576 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.495557 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:42.631342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.631273 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-94szl"] Aug 04 09:13:42.633306 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:13:42.633278 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f2546a_5a85_4f91_ad18_6d404200fd5c.slice/crio-cbb1cd7e7ae5ac2a900a153bae2be40c8e6fc3be4d2ffd081b80bfcbffb2a724 WatchSource:0}: Error finding container cbb1cd7e7ae5ac2a900a153bae2be40c8e6fc3be4d2ffd081b80bfcbffb2a724: Status 404 returned error can't find the container with id cbb1cd7e7ae5ac2a900a153bae2be40c8e6fc3be4d2ffd081b80bfcbffb2a724 Aug 04 09:13:42.941105 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.941050 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" event={"ID":"b2f2546a-5a85-4f91-ad18-6d404200fd5c","Type":"ContainerStarted","Data":"cbb1cd7e7ae5ac2a900a153bae2be40c8e6fc3be4d2ffd081b80bfcbffb2a724"} Aug 04 09:13:42.942102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.942081 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" event={"ID":"3ebe8684-4d19-4144-96d7-a66c6fdb78e7","Type":"ContainerStarted","Data":"d8036780eb5fb9f416f574838df7cd9e438603020fe6f1d8ab9072f17ec40e68"} Aug 04 09:13:42.994190 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:42.994160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:43.114274 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.114216 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xqm6t"] Aug 04 09:13:43.118928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.118910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.121797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.121773 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kxvgc\"" Aug 04 09:13:43.123287 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.123273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Aug 04 09:13:43.137606 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.137583 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xqm6t"] Aug 04 09:13:43.227813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.227737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6964\" (UniqueName: \"kubernetes.io/projected/115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6-kube-api-access-q6964\") pod \"model-serving-api-54d89bd5fc-xqm6t\" (UID: \"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6\") " pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.227994 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.227827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6-tls-certs\") pod \"model-serving-api-54d89bd5fc-xqm6t\" (UID: \"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6\") " pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.329488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.329454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6-tls-certs\") pod \"model-serving-api-54d89bd5fc-xqm6t\" (UID: \"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6\") " pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.329948 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.329586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6964\" (UniqueName: \"kubernetes.io/projected/115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6-kube-api-access-q6964\") pod \"model-serving-api-54d89bd5fc-xqm6t\" (UID: \"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6\") " pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.332876 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.332850 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6-tls-certs\") pod \"model-serving-api-54d89bd5fc-xqm6t\" (UID: \"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6\") " pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.333361 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.333329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:43.348087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.348061 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6964\" (UniqueName: \"kubernetes.io/projected/115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6-kube-api-access-q6964\") pod \"model-serving-api-54d89bd5fc-xqm6t\" (UID: \"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6\") " pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.429700 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.429626 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:43.582094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.582059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:43.633898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.633866 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xqm6t"] Aug 04 09:13:43.638214 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:13:43.638180 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115038de_d2ba_4e4f_8e89_bb7ccdcb6bd6.slice/crio-b696cb4c11aa9ca51c72fd8de93e586c70c322273a4e2f8c4a0a2399b95b180b WatchSource:0}: Error finding container b696cb4c11aa9ca51c72fd8de93e586c70c322273a4e2f8c4a0a2399b95b180b: Status 404 returned error can't find the container with id b696cb4c11aa9ca51c72fd8de93e586c70c322273a4e2f8c4a0a2399b95b180b Aug 04 09:13:43.679394 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.679362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:43.954490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:43.954413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" event={"ID":"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6","Type":"ContainerStarted","Data":"b696cb4c11aa9ca51c72fd8de93e586c70c322273a4e2f8c4a0a2399b95b180b"} Aug 04 09:13:44.712987 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:44.712943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:47.711045 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.711010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:47.802894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.802860 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:47.977878 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.977794 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" event={"ID":"115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6","Type":"ContainerStarted","Data":"e7a0f65cf302e047f99d2abaf320bdfcd85c0f03e0ab283d94062f1125ea9c7b"} Aug 04 09:13:47.978043 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.977916 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:13:47.979303 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.979279 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" event={"ID":"b2f2546a-5a85-4f91-ad18-6d404200fd5c","Type":"ContainerStarted","Data":"a5a5e96c41d7b6e2c08cd73b318ce962a24159fe62d479d26ca5a67d5e35cc34"} Aug 04 09:13:47.979441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.979327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:47.980693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:47.980672 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" event={"ID":"3ebe8684-4d19-4144-96d7-a66c6fdb78e7","Type":"ContainerStarted","Data":"f4c8e56f363418bd80fbd927136a17eafccd5b37eead63a999876eaa93416f07"} Aug 04 09:13:48.009721 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.009676 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" podStartSLOduration=1.356186579 podStartE2EDuration="5.009654623s" totalImagesPullingTime="3.653468044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:48.006942362 +0000 UTC m=+345.139151930" watchObservedRunningTime="2026-08-04 09:13:48.009654623 +0000 UTC m=+345.141864138" Aug 04 09:13:48.035382 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.035332 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" podStartSLOduration=1.3729464679999999 podStartE2EDuration="6.035312223s" totalImagesPullingTime="4.662365755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:48.032374243 +0000 UTC m=+345.164583776" watchObservedRunningTime="2026-08-04 09:13:48.035312223 +0000 UTC m=+345.167521759" Aug 04 09:13:48.065147 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.065098 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bcc56ccff-xnn4z" podStartSLOduration=1.2620393540000001 podStartE2EDuration="6.065083954s" totalImagesPullingTime="4.8030446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:48.062150562 +0000 UTC m=+345.194360093" watchObservedRunningTime="2026-08-04 09:13:48.065083954 +0000 UTC m=+345.197293485" Aug 04 09:13:48.123611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.123577 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99"] Aug 04 09:13:48.127577 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.127555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.130578 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.130555 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bw5x9\"" Aug 04 09:13:48.132712 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.132692 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 09:13:48.197140 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.197112 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:48.224271 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.224231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99"] Aug 04 09:13:48.275838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.275838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.275838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275826 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e958605-71b9-4c12-946c-a38e22ddb5c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.276029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.276029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.276029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwsdg\" (UniqueName: \"kubernetes.io/projected/5e958605-71b9-4c12-946c-a38e22ddb5c6-kube-api-access-jwsdg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.276029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.275990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.276151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.276050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.276151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.276118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377560 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e958605-71b9-4c12-946c-a38e22ddb5c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.377753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwsdg\" (UniqueName: \"kubernetes.io/projected/5e958605-71b9-4c12-946c-a38e22ddb5c6-kube-api-access-jwsdg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.377786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.378004 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.378070 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378246 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.378213 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378397 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.378376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.378447 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.378393 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e958605-71b9-4c12-946c-a38e22ddb5c6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.380008 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.379987 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.380196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.380177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.385424 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.385405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e958605-71b9-4c12-946c-a38e22ddb5c6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.385821 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.385799 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwsdg\" (UniqueName: \"kubernetes.io/projected/5e958605-71b9-4c12-946c-a38e22ddb5c6-kube-api-access-jwsdg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99\" (UID: \"5e958605-71b9-4c12-946c-a38e22ddb5c6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.438246 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.438222 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:48.500917 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.500884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:48.579403 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.579371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:48.584339 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.584309 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99"] Aug 04 09:13:48.986592 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:48.986558 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" event={"ID":"5e958605-71b9-4c12-946c-a38e22ddb5c6","Type":"ContainerStarted","Data":"580f8a7d60c64ff5a754a2f81d12c0958752cb1be5e8f35854e63daddb69dab5"} Aug 04 09:13:51.487081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.487050 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8856bf979-6lctf"] Aug 04 09:13:51.491702 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.491681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.494937 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.494913 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 09:13:51.495061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.494981 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 09:13:51.495061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.495029 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cbxtg\"" Aug 04 09:13:51.495061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.495038 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 09:13:51.501031 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.501012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8856bf979-6lctf"] Aug 04 09:13:51.504928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.504904 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-metrics-cert\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.505026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.504947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-cert\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.505026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.504998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-manager-config\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.505109 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.505046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-264xn\" (UniqueName: \"kubernetes.io/projected/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-kube-api-access-264xn\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.606020 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.605978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-metrics-cert\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.606190 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.606046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-cert\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.606190 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.606070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-manager-config\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.606190 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.606108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-264xn\" (UniqueName: \"kubernetes.io/projected/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-kube-api-access-264xn\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.606743 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.606719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-manager-config\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.608337 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.608320 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-cert\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.608465 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.608450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-metrics-cert\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.613581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.613561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-264xn\" (UniqueName: \"kubernetes.io/projected/1b34a6ca-8f30-4aeb-bd63-36aad123e6a1-kube-api-access-264xn\") pod \"lws-controller-manager-8856bf979-6lctf\" (UID: \"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1\") " pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.802284 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.802183 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:51.951369 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:51.951287 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8856bf979-6lctf"] Aug 04 09:13:51.951369 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:13:51.951292 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b34a6ca_8f30_4aeb_bd63_36aad123e6a1.slice/crio-a7b1c41ca14d219d64815fab8c12d7118cc0d41867ac640ed79ab298b08993ed WatchSource:0}: Error finding container a7b1c41ca14d219d64815fab8c12d7118cc0d41867ac640ed79ab298b08993ed: Status 404 returned error can't find the container with id a7b1c41ca14d219d64815fab8c12d7118cc0d41867ac640ed79ab298b08993ed Aug 04 09:13:52.004112 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:52.004070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" event={"ID":"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1","Type":"ContainerStarted","Data":"a7b1c41ca14d219d64815fab8c12d7118cc0d41867ac640ed79ab298b08993ed"} Aug 04 09:13:53.711926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:53.711791 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:53.885619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:53.885578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:54.016925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:54.016839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" event={"ID":"1b34a6ca-8f30-4aeb-bd63-36aad123e6a1","Type":"ContainerStarted","Data":"2171e983d11bbde8d25bc65bc8ef2ddbcd19cf58c4a0212cbf0374c1e6a5d707"} Aug 04 09:13:54.017090 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:54.017018 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:13:54.087347 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:54.087306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:54.118849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:54.118790 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" podStartSLOduration=1.656852711 podStartE2EDuration="3.118770844s" totalImagesPullingTime="1.461918133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:54.118593755 +0000 UTC m=+351.250803287" watchObservedRunningTime="2026-08-04 09:13:54.118770844 +0000 UTC m=+351.250980379" Aug 04 09:13:54.247823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:54.247787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:55.148783 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:55.148723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:55.281500 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:55.281384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:13:55.730284 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:55.730227 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:13:55.730428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:55.730316 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:13:55.730428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:55.730346 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:13:56.029025 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:56.028939 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" event={"ID":"5e958605-71b9-4c12-946c-a38e22ddb5c6","Type":"ContainerStarted","Data":"64fd6c6e55f94e7ceab225c78fe7a330bdc6d7bb8aff4f3993108721eb717d6c"} Aug 04 09:13:56.056754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:56.056708 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" podStartSLOduration=0.90720866 podStartE2EDuration="8.056674995s" totalImagesPullingTime="7.149466335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:13:56.054347496 +0000 UTC m=+353.186557028" watchObservedRunningTime="2026-08-04 09:13:56.056674995 +0000 UTC m=+353.188884589" Aug 04 09:13:56.438835 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:56.438801 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:56.443639 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:56.443612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:57.035967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:57.035930 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:57.036824 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:57.036808 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99" Aug 04 09:13:58.989167 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:58.989079 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-94szl" Aug 04 09:13:58.991632 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:13:58.991608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xqm6t" Aug 04 09:14:05.024601 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:05.024572 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8856bf979-6lctf" Aug 04 09:14:11.521137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.521101 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74575d746f-ttgrx"] Aug 04 09:14:11.533314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.533287 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.545766 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.545742 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74575d746f-ttgrx"] Aug 04 09:14:11.579786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-trusted-ca-bundle\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.579930 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-config\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.579930 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579824 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8qww\" (UniqueName: \"kubernetes.io/projected/594408d8-1ffe-40eb-92bf-0ff640a594fe-kube-api-access-v8qww\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.579930 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-serving-cert\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.579930 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-oauth-serving-cert\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.580078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-oauth-config\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.580078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.579954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-service-ca\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.665056 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.665021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:11.681225 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-config\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.681346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8qww\" (UniqueName: \"kubernetes.io/projected/594408d8-1ffe-40eb-92bf-0ff640a594fe-kube-api-access-v8qww\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.681346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-serving-cert\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.681346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-oauth-serving-cert\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.681509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-oauth-config\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.681509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-service-ca\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.681509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.681437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-trusted-ca-bundle\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.682055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.682031 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-config\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.682154 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.682125 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-service-ca\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.682206 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.682163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-oauth-serving-cert\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.682380 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.682346 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/594408d8-1ffe-40eb-92bf-0ff640a594fe-trusted-ca-bundle\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.683984 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.683959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-oauth-config\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.684078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.684040 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/594408d8-1ffe-40eb-92bf-0ff640a594fe-console-serving-cert\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.691049 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.691027 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8qww\" (UniqueName: \"kubernetes.io/projected/594408d8-1ffe-40eb-92bf-0ff640a594fe-kube-api-access-v8qww\") pod \"console-74575d746f-ttgrx\" (UID: \"594408d8-1ffe-40eb-92bf-0ff640a594fe\") " pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.758945 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.758917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:11.842944 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.842871 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:11.974475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:11.974447 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74575d746f-ttgrx"] Aug 04 09:14:11.977117 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:14:11.977084 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594408d8_1ffe_40eb_92bf_0ff640a594fe.slice/crio-a8f84252268a30c099380ac4dfbf2f518ce4b9d05441f49bf907374ddc3190e0 WatchSource:0}: Error finding container a8f84252268a30c099380ac4dfbf2f518ce4b9d05441f49bf907374ddc3190e0: Status 404 returned error can't find the container with id a8f84252268a30c099380ac4dfbf2f518ce4b9d05441f49bf907374ddc3190e0 Aug 04 09:14:12.111806 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:12.111714 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74575d746f-ttgrx" event={"ID":"594408d8-1ffe-40eb-92bf-0ff640a594fe","Type":"ContainerStarted","Data":"88e1140dd458525edebc912eef8e9e1b324767399afab94c595aa4ffaea95676"} Aug 04 09:14:12.111806 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:12.111774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74575d746f-ttgrx" event={"ID":"594408d8-1ffe-40eb-92bf-0ff640a594fe","Type":"ContainerStarted","Data":"a8f84252268a30c099380ac4dfbf2f518ce4b9d05441f49bf907374ddc3190e0"} Aug 04 09:14:12.133369 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:12.133323 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74575d746f-ttgrx" podStartSLOduration=1.133307785 podStartE2EDuration="1.133307785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:14:12.131611728 +0000 UTC m=+369.263821274" watchObservedRunningTime="2026-08-04 09:14:12.133307785 +0000 UTC m=+369.265517317" Aug 04 09:14:21.843823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:21.843779 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:21.844290 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:21.843865 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:21.848849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:21.848826 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:22.163478 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:22.163447 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74575d746f-ttgrx" Aug 04 09:14:22.218032 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:22.217997 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8474dd7d9c-7x6j9"] Aug 04 09:14:22.508050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:22.507973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:22.609403 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:22.609370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:40.724414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:40.724379 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:43.000484 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:43.000442 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:43.159417 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:43.159380 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:46.395061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.395025 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d"] Aug 04 09:14:46.403552 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.403527 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.407196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.407174 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v49jl\"" Aug 04 09:14:46.407358 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.407222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:14:46.407358 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.407222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:14:46.408219 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.408190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d"] Aug 04 09:14:46.485247 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.485192 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p"] Aug 04 09:14:46.486515 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.486488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.486803 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.486776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frh6h\" (UniqueName: \"kubernetes.io/projected/8db740aa-9d4f-4f25-a94c-54f444daba23-kube-api-access-frh6h\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.486976 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.486882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.489019 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.489003 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.497620 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.497597 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p"] Aug 04 09:14:46.585903 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.585870 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4"] Aug 04 09:14:46.588032 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.588154 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588052 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8259\" (UniqueName: \"kubernetes.io/projected/2dbafb77-ab19-4b51-b3e1-f4538b12b816-kube-api-access-r8259\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.588154 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.588154 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588115 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frh6h\" (UniqueName: \"kubernetes.io/projected/8db740aa-9d4f-4f25-a94c-54f444daba23-kube-api-access-frh6h\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.588154 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.588342 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.588452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.588501 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.588456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.589653 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.589639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.596984 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.596963 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frh6h\" (UniqueName: \"kubernetes.io/projected/8db740aa-9d4f-4f25-a94c-54f444daba23-kube-api-access-frh6h\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.599068 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.599049 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4"] Aug 04 09:14:46.688290 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.688177 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd"] Aug 04 09:14:46.689383 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8259\" (UniqueName: \"kubernetes.io/projected/2dbafb77-ab19-4b51-b3e1-f4538b12b816-kube-api-access-r8259\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.689526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.689526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.689526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.689526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8wbj\" (UniqueName: \"kubernetes.io/projected/bdb0a31d-ae75-4c12-b829-e85e21efc829-kube-api-access-c8wbj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.689736 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.689796 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.689845 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.689796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.692198 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.692179 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.697787 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.697763 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8259\" (UniqueName: \"kubernetes.io/projected/2dbafb77-ab19-4b51-b3e1-f4538b12b816-kube-api-access-r8259\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.701227 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.701206 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd"] Aug 04 09:14:46.715927 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.715908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.790699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv2dq\" (UniqueName: \"kubernetes.io/projected/7a6c00e7-36f2-4355-915e-1641e71a895e-kube-api-access-pv2dq\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.790761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.790838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8wbj\" (UniqueName: \"kubernetes.io/projected/bdb0a31d-ae75-4c12-b829-e85e21efc829-kube-api-access-c8wbj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.790877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.790901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.790953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.791360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.791939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.791892 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.797684 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.797451 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:46.800763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.800706 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8wbj\" (UniqueName: \"kubernetes.io/projected/bdb0a31d-ae75-4c12-b829-e85e21efc829-kube-api-access-c8wbj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.868960 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.868936 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d"] Aug 04 09:14:46.870388 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:14:46.870361 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db740aa_9d4f_4f25_a94c_54f444daba23.slice/crio-5bd651a5b922f196e3dbc241296561dfe50f70b4348c2849027a09a7ac51e49a WatchSource:0}: Error finding container 5bd651a5b922f196e3dbc241296561dfe50f70b4348c2849027a09a7ac51e49a: Status 404 returned error can't find the container with id 5bd651a5b922f196e3dbc241296561dfe50f70b4348c2849027a09a7ac51e49a Aug 04 09:14:46.892229 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.892194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv2dq\" (UniqueName: \"kubernetes.io/projected/7a6c00e7-36f2-4355-915e-1641e71a895e-kube-api-access-pv2dq\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.892388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.892291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.892475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.892389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.892816 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.892792 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.892816 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.892815 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.900689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.900664 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv2dq\" (UniqueName: \"kubernetes.io/projected/7a6c00e7-36f2-4355-915e-1641e71a895e-kube-api-access-pv2dq\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:46.908035 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.907516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:46.939292 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:46.939203 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p"] Aug 04 09:14:46.939889 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:14:46.939849 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dbafb77_ab19_4b51_b3e1_f4538b12b816.slice/crio-fc3943aa430d2c5580f60a36d1a6220e221c1897a2bd0d35bf76278e8e22efbf WatchSource:0}: Error finding container fc3943aa430d2c5580f60a36d1a6220e221c1897a2bd0d35bf76278e8e22efbf: Status 404 returned error can't find the container with id fc3943aa430d2c5580f60a36d1a6220e221c1897a2bd0d35bf76278e8e22efbf Aug 04 09:14:47.010602 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.010520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:47.065000 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.064963 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4"] Aug 04 09:14:47.066091 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:14:47.066060 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb0a31d_ae75_4c12_b829_e85e21efc829.slice/crio-cdf3273bba4392b00b6a68961705d06353fe8666110a9bf149c1cb90a1de7bfc WatchSource:0}: Error finding container cdf3273bba4392b00b6a68961705d06353fe8666110a9bf149c1cb90a1de7bfc: Status 404 returned error can't find the container with id cdf3273bba4392b00b6a68961705d06353fe8666110a9bf149c1cb90a1de7bfc Aug 04 09:14:47.156993 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.156968 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd"] Aug 04 09:14:47.159149 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:14:47.159126 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6c00e7_36f2_4355_915e_1641e71a895e.slice/crio-350d54078a2be73e3fb2b61b716ca8a855cb3ab22f91385f0f41fd8d8c204fa6 WatchSource:0}: Error finding container 350d54078a2be73e3fb2b61b716ca8a855cb3ab22f91385f0f41fd8d8c204fa6: Status 404 returned error can't find the container with id 350d54078a2be73e3fb2b61b716ca8a855cb3ab22f91385f0f41fd8d8c204fa6 Aug 04 09:14:47.237480 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.237445 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8474dd7d9c-7x6j9" podUID="70506efe-201b-4fa1-a1e7-8aa06e72635e" containerName="console" containerID="cri-o://f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d" gracePeriod=15 Aug 04 09:14:47.283925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.283897 2562 generic.go:356] "Generic (PLEG): container finished" podID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerID="77c42f19def824e56cb7b3d0517ff03847a622ea18394c96b88be3535bae1c31" exitCode=0 Aug 04 09:14:47.284040 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.283986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" event={"ID":"bdb0a31d-ae75-4c12-b829-e85e21efc829","Type":"ContainerDied","Data":"77c42f19def824e56cb7b3d0517ff03847a622ea18394c96b88be3535bae1c31"} Aug 04 09:14:47.284101 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.284047 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" event={"ID":"bdb0a31d-ae75-4c12-b829-e85e21efc829","Type":"ContainerStarted","Data":"cdf3273bba4392b00b6a68961705d06353fe8666110a9bf149c1cb90a1de7bfc"} Aug 04 09:14:47.285526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.285417 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" event={"ID":"7a6c00e7-36f2-4355-915e-1641e71a895e","Type":"ContainerStarted","Data":"74cd24fe43d41da48a13f71f5eb34ea0f92dda65b0b7719d792200a468d2b57e"} Aug 04 09:14:47.285526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.285468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" event={"ID":"7a6c00e7-36f2-4355-915e-1641e71a895e","Type":"ContainerStarted","Data":"350d54078a2be73e3fb2b61b716ca8a855cb3ab22f91385f0f41fd8d8c204fa6"} Aug 04 09:14:47.286912 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.286892 2562 generic.go:356] "Generic (PLEG): container finished" podID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerID="a92747e3cbe9858ff0ef7f4c71ec7d4e340d243c182bd6f8388d45eb673eba5c" exitCode=0 Aug 04 09:14:47.286999 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.286951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" event={"ID":"2dbafb77-ab19-4b51-b3e1-f4538b12b816","Type":"ContainerDied","Data":"a92747e3cbe9858ff0ef7f4c71ec7d4e340d243c182bd6f8388d45eb673eba5c"} Aug 04 09:14:47.286999 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.286990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" event={"ID":"2dbafb77-ab19-4b51-b3e1-f4538b12b816","Type":"ContainerStarted","Data":"fc3943aa430d2c5580f60a36d1a6220e221c1897a2bd0d35bf76278e8e22efbf"} Aug 04 09:14:47.288496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.288479 2562 generic.go:356] "Generic (PLEG): container finished" podID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerID="8a821e14d4eadb46593abfdb766ec7bf9851679c44fc3eea5b5504714ebfa1ad" exitCode=0 Aug 04 09:14:47.288564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.288520 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" event={"ID":"8db740aa-9d4f-4f25-a94c-54f444daba23","Type":"ContainerDied","Data":"8a821e14d4eadb46593abfdb766ec7bf9851679c44fc3eea5b5504714ebfa1ad"} Aug 04 09:14:47.288564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.288545 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" event={"ID":"8db740aa-9d4f-4f25-a94c-54f444daba23","Type":"ContainerStarted","Data":"5bd651a5b922f196e3dbc241296561dfe50f70b4348c2849027a09a7ac51e49a"} Aug 04 09:14:47.465013 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.464990 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:14:47.497671 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497646 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-service-ca\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.497800 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-config\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.497837 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497812 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-oauth-config\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.497884 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497844 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ktm9\" (UniqueName: \"kubernetes.io/projected/70506efe-201b-4fa1-a1e7-8aa06e72635e-kube-api-access-2ktm9\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.497941 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497882 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-serving-cert\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.497941 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497912 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-trusted-ca-bundle\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.498043 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.497960 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-oauth-serving-cert\") pod \"70506efe-201b-4fa1-a1e7-8aa06e72635e\" (UID: \"70506efe-201b-4fa1-a1e7-8aa06e72635e\") " Aug 04 09:14:47.498099 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.498052 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-service-ca" (OuterVolumeSpecName: "service-ca") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:14:47.498302 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.498283 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:47.498409 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.498370 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:14:47.498409 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.498390 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:14:47.498515 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.498421 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-config" (OuterVolumeSpecName: "console-config") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:14:47.499902 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.499885 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70506efe-201b-4fa1-a1e7-8aa06e72635e-kube-api-access-2ktm9" (OuterVolumeSpecName: "kube-api-access-2ktm9") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "kube-api-access-2ktm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:14:47.500319 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.500301 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:14:47.500384 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.500327 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70506efe-201b-4fa1-a1e7-8aa06e72635e" (UID: "70506efe-201b-4fa1-a1e7-8aa06e72635e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:14:47.599478 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.599442 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:47.599478 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.599480 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-trusted-ca-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:47.599657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.599496 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-oauth-serving-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:47.599657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.599512 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:47.599657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.599526 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70506efe-201b-4fa1-a1e7-8aa06e72635e-console-oauth-config\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:47.599657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:47.599539 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ktm9\" (UniqueName: \"kubernetes.io/projected/70506efe-201b-4fa1-a1e7-8aa06e72635e-kube-api-access-2ktm9\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:48.295382 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.295309 2562 generic.go:356] "Generic (PLEG): container finished" podID="70506efe-201b-4fa1-a1e7-8aa06e72635e" containerID="f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d" exitCode=0 Aug 04 09:14:48.295382 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.295359 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8474dd7d9c-7x6j9" Aug 04 09:14:48.295575 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.295406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474dd7d9c-7x6j9" event={"ID":"70506efe-201b-4fa1-a1e7-8aa06e72635e","Type":"ContainerDied","Data":"f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d"} Aug 04 09:14:48.295575 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.295467 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8474dd7d9c-7x6j9" event={"ID":"70506efe-201b-4fa1-a1e7-8aa06e72635e","Type":"ContainerDied","Data":"78660bc1de2c4de54624db16f721bb3efed506ce1ba19145cf3e87b4adf98bdc"} Aug 04 09:14:48.295575 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.295497 2562 scope.go:117] "RemoveContainer" containerID="f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d" Aug 04 09:14:48.297345 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.297325 2562 generic.go:356] "Generic (PLEG): container finished" podID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerID="1070b4676169b1f95b6bf3ad59c8c00409ba9e9b6b33c916672be6a365296e62" exitCode=0 Aug 04 09:14:48.297462 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.297400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" event={"ID":"bdb0a31d-ae75-4c12-b829-e85e21efc829","Type":"ContainerDied","Data":"1070b4676169b1f95b6bf3ad59c8c00409ba9e9b6b33c916672be6a365296e62"} Aug 04 09:14:48.299146 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.299126 2562 generic.go:356] "Generic (PLEG): container finished" podID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerID="74cd24fe43d41da48a13f71f5eb34ea0f92dda65b0b7719d792200a468d2b57e" exitCode=0 Aug 04 09:14:48.299322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.299193 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" event={"ID":"7a6c00e7-36f2-4355-915e-1641e71a895e","Type":"ContainerDied","Data":"74cd24fe43d41da48a13f71f5eb34ea0f92dda65b0b7719d792200a468d2b57e"} Aug 04 09:14:48.300977 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.300954 2562 generic.go:356] "Generic (PLEG): container finished" podID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerID="bd6cde0141065c2d6e9bbefedd71bf08b47aaebf1e033328c8d63ed394fb524f" exitCode=0 Aug 04 09:14:48.301078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.300982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" event={"ID":"8db740aa-9d4f-4f25-a94c-54f444daba23","Type":"ContainerDied","Data":"bd6cde0141065c2d6e9bbefedd71bf08b47aaebf1e033328c8d63ed394fb524f"} Aug 04 09:14:48.305226 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.305049 2562 scope.go:117] "RemoveContainer" containerID="f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d" Aug 04 09:14:48.305497 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:14:48.305470 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d\": container with ID starting with f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d not found: ID does not exist" containerID="f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d" Aug 04 09:14:48.305577 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.305506 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d"} err="failed to get container status \"f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d\": rpc error: code = NotFound desc = could not find container \"f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d\": container with ID starting with f5bb7481126a1001d5c41eeb12215ebd03d1f906e8d783279aa478fda579752d not found: ID does not exist" Aug 04 09:14:48.349467 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.349443 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8474dd7d9c-7x6j9"] Aug 04 09:14:48.355839 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:48.355819 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8474dd7d9c-7x6j9"] Aug 04 09:14:49.309761 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.309721 2562 generic.go:356] "Generic (PLEG): container finished" podID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerID="0c553ebc735bb83c83df12ef46e4598acb4ae5caf52cca53293cf8639e0db12f" exitCode=0 Aug 04 09:14:49.310131 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.309820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" event={"ID":"bdb0a31d-ae75-4c12-b829-e85e21efc829","Type":"ContainerDied","Data":"0c553ebc735bb83c83df12ef46e4598acb4ae5caf52cca53293cf8639e0db12f"} Aug 04 09:14:49.311657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.311626 2562 generic.go:356] "Generic (PLEG): container finished" podID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerID="170d936763dbcf8d970338d0b37f83a8738557c59246eb8d85b232ade306989c" exitCode=0 Aug 04 09:14:49.311797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.311688 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" event={"ID":"7a6c00e7-36f2-4355-915e-1641e71a895e","Type":"ContainerDied","Data":"170d936763dbcf8d970338d0b37f83a8738557c59246eb8d85b232ade306989c"} Aug 04 09:14:49.313572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.313550 2562 generic.go:356] "Generic (PLEG): container finished" podID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerID="8b9082a33776083ad7ff0640e24590777da5a8abae50a14aac37987eb4fe277e" exitCode=0 Aug 04 09:14:49.313673 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.313580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" event={"ID":"2dbafb77-ab19-4b51-b3e1-f4538b12b816","Type":"ContainerDied","Data":"8b9082a33776083ad7ff0640e24590777da5a8abae50a14aac37987eb4fe277e"} Aug 04 09:14:49.315920 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.315900 2562 generic.go:356] "Generic (PLEG): container finished" podID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerID="8e6b37f8f5552bde92c4e83ee6165eddb717f3a6ae813baf8a48607129823618" exitCode=0 Aug 04 09:14:49.315995 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.315923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" event={"ID":"8db740aa-9d4f-4f25-a94c-54f444daba23","Type":"ContainerDied","Data":"8e6b37f8f5552bde92c4e83ee6165eddb717f3a6ae813baf8a48607129823618"} Aug 04 09:14:49.421837 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:49.421803 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70506efe-201b-4fa1-a1e7-8aa06e72635e" path="/var/lib/kubelet/pods/70506efe-201b-4fa1-a1e7-8aa06e72635e/volumes" Aug 04 09:14:50.323220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.323187 2562 generic.go:356] "Generic (PLEG): container finished" podID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerID="5d1500500a2359879003a0e4eec7a95c6bad4aeef22b51f80571b5c84d754c7e" exitCode=0 Aug 04 09:14:50.323667 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.323280 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" event={"ID":"7a6c00e7-36f2-4355-915e-1641e71a895e","Type":"ContainerDied","Data":"5d1500500a2359879003a0e4eec7a95c6bad4aeef22b51f80571b5c84d754c7e"} Aug 04 09:14:50.325104 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.325084 2562 generic.go:356] "Generic (PLEG): container finished" podID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerID="4623bc4fbede30a93656a62c709756b9dca91552159c377e93959fdbe7218ecb" exitCode=0 Aug 04 09:14:50.325243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.325164 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" event={"ID":"2dbafb77-ab19-4b51-b3e1-f4538b12b816","Type":"ContainerDied","Data":"4623bc4fbede30a93656a62c709756b9dca91552159c377e93959fdbe7218ecb"} Aug 04 09:14:50.470767 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.470745 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:50.498786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.498765 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:50.527548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.527522 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-util\") pod \"bdb0a31d-ae75-4c12-b829-e85e21efc829\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " Aug 04 09:14:50.527654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.527588 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8wbj\" (UniqueName: \"kubernetes.io/projected/bdb0a31d-ae75-4c12-b829-e85e21efc829-kube-api-access-c8wbj\") pod \"bdb0a31d-ae75-4c12-b829-e85e21efc829\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " Aug 04 09:14:50.527691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.527659 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frh6h\" (UniqueName: \"kubernetes.io/projected/8db740aa-9d4f-4f25-a94c-54f444daba23-kube-api-access-frh6h\") pod \"8db740aa-9d4f-4f25-a94c-54f444daba23\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " Aug 04 09:14:50.527727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.527689 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-util\") pod \"8db740aa-9d4f-4f25-a94c-54f444daba23\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " Aug 04 09:14:50.527727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.527714 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-bundle\") pod \"bdb0a31d-ae75-4c12-b829-e85e21efc829\" (UID: \"bdb0a31d-ae75-4c12-b829-e85e21efc829\") " Aug 04 09:14:50.527828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.527804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-bundle\") pod \"8db740aa-9d4f-4f25-a94c-54f444daba23\" (UID: \"8db740aa-9d4f-4f25-a94c-54f444daba23\") " Aug 04 09:14:50.528542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.528487 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-bundle" (OuterVolumeSpecName: "bundle") pod "8db740aa-9d4f-4f25-a94c-54f444daba23" (UID: "8db740aa-9d4f-4f25-a94c-54f444daba23"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:50.528815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.528782 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-bundle" (OuterVolumeSpecName: "bundle") pod "bdb0a31d-ae75-4c12-b829-e85e21efc829" (UID: "bdb0a31d-ae75-4c12-b829-e85e21efc829"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:50.530276 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.530232 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db740aa-9d4f-4f25-a94c-54f444daba23-kube-api-access-frh6h" (OuterVolumeSpecName: "kube-api-access-frh6h") pod "8db740aa-9d4f-4f25-a94c-54f444daba23" (UID: "8db740aa-9d4f-4f25-a94c-54f444daba23"). InnerVolumeSpecName "kube-api-access-frh6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:14:50.530418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.530290 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb0a31d-ae75-4c12-b829-e85e21efc829-kube-api-access-c8wbj" (OuterVolumeSpecName: "kube-api-access-c8wbj") pod "bdb0a31d-ae75-4c12-b829-e85e21efc829" (UID: "bdb0a31d-ae75-4c12-b829-e85e21efc829"). InnerVolumeSpecName "kube-api-access-c8wbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:14:50.536067 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.536041 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-util" (OuterVolumeSpecName: "util") pod "bdb0a31d-ae75-4c12-b829-e85e21efc829" (UID: "bdb0a31d-ae75-4c12-b829-e85e21efc829"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:50.536384 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.536363 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-util" (OuterVolumeSpecName: "util") pod "8db740aa-9d4f-4f25-a94c-54f444daba23" (UID: "8db740aa-9d4f-4f25-a94c-54f444daba23"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:50.628931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.628904 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:50.628931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.628927 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8wbj\" (UniqueName: \"kubernetes.io/projected/bdb0a31d-ae75-4c12-b829-e85e21efc829-kube-api-access-c8wbj\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:50.629064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.628937 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frh6h\" (UniqueName: \"kubernetes.io/projected/8db740aa-9d4f-4f25-a94c-54f444daba23-kube-api-access-frh6h\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:50.629064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.628945 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:50.629064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.628953 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdb0a31d-ae75-4c12-b829-e85e21efc829-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:50.629064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:50.628961 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db740aa-9d4f-4f25-a94c-54f444daba23-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:51.334025 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.333989 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" Aug 04 09:14:51.334025 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.334011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537bcz4" event={"ID":"bdb0a31d-ae75-4c12-b829-e85e21efc829","Type":"ContainerDied","Data":"cdf3273bba4392b00b6a68961705d06353fe8666110a9bf149c1cb90a1de7bfc"} Aug 04 09:14:51.334563 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.334055 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdf3273bba4392b00b6a68961705d06353fe8666110a9bf149c1cb90a1de7bfc" Aug 04 09:14:51.335827 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.335800 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" event={"ID":"8db740aa-9d4f-4f25-a94c-54f444daba23","Type":"ContainerDied","Data":"5bd651a5b922f196e3dbc241296561dfe50f70b4348c2849027a09a7ac51e49a"} Aug 04 09:14:51.335827 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.335825 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cfm47d" Aug 04 09:14:51.335977 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.335844 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bd651a5b922f196e3dbc241296561dfe50f70b4348c2849027a09a7ac51e49a" Aug 04 09:14:51.490532 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.490505 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:51.492490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.492473 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:51.537563 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.537533 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-util\") pod \"7a6c00e7-36f2-4355-915e-1641e71a895e\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " Aug 04 09:14:51.537705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.537590 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-bundle\") pod \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " Aug 04 09:14:51.537705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.537635 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-util\") pod \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " Aug 04 09:14:51.537705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.537664 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv2dq\" (UniqueName: \"kubernetes.io/projected/7a6c00e7-36f2-4355-915e-1641e71a895e-kube-api-access-pv2dq\") pod \"7a6c00e7-36f2-4355-915e-1641e71a895e\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " Aug 04 09:14:51.537873 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.537703 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8259\" (UniqueName: \"kubernetes.io/projected/2dbafb77-ab19-4b51-b3e1-f4538b12b816-kube-api-access-r8259\") pod \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\" (UID: \"2dbafb77-ab19-4b51-b3e1-f4538b12b816\") " Aug 04 09:14:51.537873 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.537781 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-bundle\") pod \"7a6c00e7-36f2-4355-915e-1641e71a895e\" (UID: \"7a6c00e7-36f2-4355-915e-1641e71a895e\") " Aug 04 09:14:51.538157 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.538093 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-bundle" (OuterVolumeSpecName: "bundle") pod "2dbafb77-ab19-4b51-b3e1-f4538b12b816" (UID: "2dbafb77-ab19-4b51-b3e1-f4538b12b816"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:51.538620 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.538589 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-bundle" (OuterVolumeSpecName: "bundle") pod "7a6c00e7-36f2-4355-915e-1641e71a895e" (UID: "7a6c00e7-36f2-4355-915e-1641e71a895e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:51.540207 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.540156 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dbafb77-ab19-4b51-b3e1-f4538b12b816-kube-api-access-r8259" (OuterVolumeSpecName: "kube-api-access-r8259") pod "2dbafb77-ab19-4b51-b3e1-f4538b12b816" (UID: "2dbafb77-ab19-4b51-b3e1-f4538b12b816"). InnerVolumeSpecName "kube-api-access-r8259". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:14:51.540207 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.540170 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a6c00e7-36f2-4355-915e-1641e71a895e-kube-api-access-pv2dq" (OuterVolumeSpecName: "kube-api-access-pv2dq") pod "7a6c00e7-36f2-4355-915e-1641e71a895e" (UID: "7a6c00e7-36f2-4355-915e-1641e71a895e"). InnerVolumeSpecName "kube-api-access-pv2dq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:14:51.543878 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.543855 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-util" (OuterVolumeSpecName: "util") pod "2dbafb77-ab19-4b51-b3e1-f4538b12b816" (UID: "2dbafb77-ab19-4b51-b3e1-f4538b12b816"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:51.546768 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.546746 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-util" (OuterVolumeSpecName: "util") pod "7a6c00e7-36f2-4355-915e-1641e71a895e" (UID: "7a6c00e7-36f2-4355-915e-1641e71a895e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:14:51.638988 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.638962 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8259\" (UniqueName: \"kubernetes.io/projected/2dbafb77-ab19-4b51-b3e1-f4538b12b816-kube-api-access-r8259\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:51.638988 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.638988 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:51.639128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.638997 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a6c00e7-36f2-4355-915e-1641e71a895e-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:51.639128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.639005 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-bundle\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:51.639128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.639012 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dbafb77-ab19-4b51-b3e1-f4538b12b816-util\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:51.639128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:51.639019 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv2dq\" (UniqueName: \"kubernetes.io/projected/7a6c00e7-36f2-4355-915e-1641e71a895e-kube-api-access-pv2dq\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:14:52.343510 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:52.343484 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" Aug 04 09:14:52.343990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:52.343470 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbrkrdd" event={"ID":"7a6c00e7-36f2-4355-915e-1641e71a895e","Type":"ContainerDied","Data":"350d54078a2be73e3fb2b61b716ca8a855cb3ab22f91385f0f41fd8d8c204fa6"} Aug 04 09:14:52.343990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:52.343605 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="350d54078a2be73e3fb2b61b716ca8a855cb3ab22f91385f0f41fd8d8c204fa6" Aug 04 09:14:52.345360 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:52.345334 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" Aug 04 09:14:52.345591 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:52.345367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0rjd2p" event={"ID":"2dbafb77-ab19-4b51-b3e1-f4538b12b816","Type":"ContainerDied","Data":"fc3943aa430d2c5580f60a36d1a6220e221c1897a2bd0d35bf76278e8e22efbf"} Aug 04 09:14:52.345591 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:52.345397 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc3943aa430d2c5580f60a36d1a6220e221c1897a2bd0d35bf76278e8e22efbf" Aug 04 09:14:55.863237 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:55.863205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:58.457173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:58.457141 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:58.840748 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:58.840671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:14:59.250441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:14:59.250403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:00.246933 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:00.246900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:01.949036 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:01.949004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:02.125202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:02.125173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:02.650680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:02.650647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:03.079320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:03.079210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:03.971107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:03.971070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:04.080159 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:04.080114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:05.655580 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:05.655533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:07.492952 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:07.492918 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:08.736831 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:08.736796 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:08.840053 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:08.840025 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:09.586535 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:09.586505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:09.705854 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:09.705824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:13.303341 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:13.303303 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:13.404511 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:13.404476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:13.617482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:13.617396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:13.718025 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:13.717994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:14.457442 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:14.457410 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:14.534091 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:14.534062 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:22.138315 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:22.138276 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:23.009965 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:23.009903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:25.118870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:25.118833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:25.222033 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:25.221956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:27.710173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:27.710134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:27.810869 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:27.810833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:32.295899 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:32.295859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:32.401472 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:32.401436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:32.496132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:32.496091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:38.736031 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:38.735997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:41.159948 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:41.159907 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:41.267206 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:41.267170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:44.117084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:44.117045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:44.217499 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:44.217456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:46.726401 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:46.726365 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:46.826911 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:46.826871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:52.115081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:52.115046 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:52.216345 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:52.216312 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:52.513932 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:52.513898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:52.615115 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:52.615083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:52.928270 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:52.928214 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:53.028622 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:53.028594 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:53.268229 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:53.268148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:53.687363 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:53.687327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:58.182269 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.182231 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:58.298761 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.298731 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:58.358159 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.358120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:58.992425 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992351 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-g6r7j"] Aug 04 09:15:58.992721 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992710 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="util" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992723 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="util" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992731 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="util" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992737 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="util" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992746 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="util" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992751 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="util" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992758 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="extract" Aug 04 09:15:58.992764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992763 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992770 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992775 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992784 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992790 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992799 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992804 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992810 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992816 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992822 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992827 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992837 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="util" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992843 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="util" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992850 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70506efe-201b-4fa1-a1e7-8aa06e72635e" containerName="console" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992855 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="70506efe-201b-4fa1-a1e7-8aa06e72635e" containerName="console" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992860 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992865 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="pull" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992873 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992878 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992928 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dbafb77-ab19-4b51-b3e1-f4538b12b816" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992936 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db740aa-9d4f-4f25-a94c-54f444daba23" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992943 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6c00e7-36f2-4355-915e-1641e71a895e" containerName="extract" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992950 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="70506efe-201b-4fa1-a1e7-8aa06e72635e" containerName="console" Aug 04 09:15:58.992978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.992956 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdb0a31d-ae75-4c12-b829-e85e21efc829" containerName="extract" Aug 04 09:15:58.995985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.995962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:15:58.998367 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.998348 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 09:15:58.999226 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.999209 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 09:15:58.999355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:58.999231 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n9gbw\"" Aug 04 09:15:59.003961 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.003941 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-g6r7j"] Aug 04 09:15:59.023799 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.023778 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:59.116822 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.116792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpnsf\" (UniqueName: \"kubernetes.io/projected/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d-kube-api-access-fpnsf\") pod \"authorino-b54df94b4-g6r7j\" (UID: \"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d\") " pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:15:59.118031 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.117990 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:15:59.138263 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.138229 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qct98"] Aug 04 09:15:59.141542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.141526 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:15:59.150663 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.150639 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qct98"] Aug 04 09:15:59.217412 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.217382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpnsf\" (UniqueName: \"kubernetes.io/projected/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d-kube-api-access-fpnsf\") pod \"authorino-b54df94b4-g6r7j\" (UID: \"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d\") " pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:15:59.217750 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.217428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwmwb\" (UniqueName: \"kubernetes.io/projected/d808d816-698e-48f4-ab77-32ad46c48847-kube-api-access-gwmwb\") pod \"authorino-c548b5d7d-qct98\" (UID: \"d808d816-698e-48f4-ab77-32ad46c48847\") " pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:15:59.226745 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.226725 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpnsf\" (UniqueName: \"kubernetes.io/projected/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d-kube-api-access-fpnsf\") pod \"authorino-b54df94b4-g6r7j\" (UID: \"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d\") " pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:15:59.305716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.305629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:15:59.317974 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.317948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwmwb\" (UniqueName: \"kubernetes.io/projected/d808d816-698e-48f4-ab77-32ad46c48847-kube-api-access-gwmwb\") pod \"authorino-c548b5d7d-qct98\" (UID: \"d808d816-698e-48f4-ab77-32ad46c48847\") " pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:15:59.326095 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.326068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwmwb\" (UniqueName: \"kubernetes.io/projected/d808d816-698e-48f4-ab77-32ad46c48847-kube-api-access-gwmwb\") pod \"authorino-c548b5d7d-qct98\" (UID: \"d808d816-698e-48f4-ab77-32ad46c48847\") " pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:15:59.438873 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.438846 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-g6r7j"] Aug 04 09:15:59.440852 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:15:59.440824 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b2470d3_ff10_4e5d_a3b4_e2f009eb0a8d.slice/crio-275c949ab058f01a0c630a42b70a57bbec1be8f2fae55b64b080aec9291fbda2 WatchSource:0}: Error finding container 275c949ab058f01a0c630a42b70a57bbec1be8f2fae55b64b080aec9291fbda2: Status 404 returned error can't find the container with id 275c949ab058f01a0c630a42b70a57bbec1be8f2fae55b64b080aec9291fbda2 Aug 04 09:15:59.451705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.451684 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:15:59.581192 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.581164 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qct98"] Aug 04 09:15:59.582962 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:15:59.582930 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd808d816_698e_48f4_ab77_32ad46c48847.slice/crio-ec5ea4d7bcf8e78edc6b2a5c33b027fa08bd6f99b7b2bd78269fafcc2374e561 WatchSource:0}: Error finding container ec5ea4d7bcf8e78edc6b2a5c33b027fa08bd6f99b7b2bd78269fafcc2374e561: Status 404 returned error can't find the container with id ec5ea4d7bcf8e78edc6b2a5c33b027fa08bd6f99b7b2bd78269fafcc2374e561 Aug 04 09:15:59.681078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.681040 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-g6r7j" event={"ID":"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d","Type":"ContainerStarted","Data":"275c949ab058f01a0c630a42b70a57bbec1be8f2fae55b64b080aec9291fbda2"} Aug 04 09:15:59.682085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:15:59.682068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qct98" event={"ID":"d808d816-698e-48f4-ab77-32ad46c48847","Type":"ContainerStarted","Data":"ec5ea4d7bcf8e78edc6b2a5c33b027fa08bd6f99b7b2bd78269fafcc2374e561"} Aug 04 09:16:00.918076 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:00.918034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:01.020440 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:01.020403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:02.705061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:02.705021 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-g6r7j" event={"ID":"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d","Type":"ContainerStarted","Data":"d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75"} Aug 04 09:16:02.706533 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:02.706507 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qct98" event={"ID":"d808d816-698e-48f4-ab77-32ad46c48847","Type":"ContainerStarted","Data":"444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42"} Aug 04 09:16:02.725372 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:02.725328 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-g6r7j" podStartSLOduration=1.646238649 podStartE2EDuration="4.725315992s" totalImagesPullingTime="3.079077343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:02.722797096 +0000 UTC m=+479.855006628" watchObservedRunningTime="2026-08-04 09:16:02.725315992 +0000 UTC m=+479.857525523" Aug 04 09:16:02.743100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:02.743049 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-qct98" podStartSLOduration=0.795040026 podStartE2EDuration="3.743036974s" totalImagesPullingTime="2.947996948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:02.742078294 +0000 UTC m=+479.874287827" watchObservedRunningTime="2026-08-04 09:16:02.743036974 +0000 UTC m=+479.875246505" Aug 04 09:16:02.770533 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:02.770463 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-g6r7j"] Aug 04 09:16:03.973355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:03.973321 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:04.070990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:04.070951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:04.717204 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:04.717165 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-g6r7j" podUID="7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" containerName="authorino" containerID="cri-o://d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75" gracePeriod=30 Aug 04 09:16:04.955684 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:04.955656 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:16:05.077456 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.077372 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpnsf\" (UniqueName: \"kubernetes.io/projected/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d-kube-api-access-fpnsf\") pod \"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d\" (UID: \"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d\") " Aug 04 09:16:05.079488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.079462 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d-kube-api-access-fpnsf" (OuterVolumeSpecName: "kube-api-access-fpnsf") pod "7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" (UID: "7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d"). InnerVolumeSpecName "kube-api-access-fpnsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:16:05.178048 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.178015 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpnsf\" (UniqueName: \"kubernetes.io/projected/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d-kube-api-access-fpnsf\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:16:05.724243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.724158 2562 generic.go:356] "Generic (PLEG): container finished" podID="7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" containerID="d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75" exitCode=0 Aug 04 09:16:05.724243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.724210 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-g6r7j" Aug 04 09:16:05.724243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.724233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-g6r7j" event={"ID":"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d","Type":"ContainerDied","Data":"d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75"} Aug 04 09:16:05.724490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.724281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-g6r7j" event={"ID":"7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d","Type":"ContainerDied","Data":"275c949ab058f01a0c630a42b70a57bbec1be8f2fae55b64b080aec9291fbda2"} Aug 04 09:16:05.724490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.724300 2562 scope.go:117] "RemoveContainer" containerID="d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75" Aug 04 09:16:05.733058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.733041 2562 scope.go:117] "RemoveContainer" containerID="d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75" Aug 04 09:16:05.733338 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:16:05.733318 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75\": container with ID starting with d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75 not found: ID does not exist" containerID="d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75" Aug 04 09:16:05.733406 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.733346 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75"} err="failed to get container status \"d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75\": rpc error: code = NotFound desc = could not find container \"d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75\": container with ID starting with d4d94153d823749c41cd0cb3ef1e904e19a8cc48073b2c5a27103c4e464f7c75 not found: ID does not exist" Aug 04 09:16:05.742213 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.742186 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-g6r7j"] Aug 04 09:16:05.744187 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:05.744168 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-g6r7j"] Aug 04 09:16:07.416416 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:07.416386 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" path="/var/lib/kubelet/pods/7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d/volumes" Aug 04 09:16:14.347580 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:14.347544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:14.436828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:14.436801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:14.787458 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:14.787407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:26.687644 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.687609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:26.902423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.902389 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-cpvn7"] Aug 04 09:16:26.902757 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.902745 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" containerName="authorino" Aug 04 09:16:26.902812 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.902759 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" containerName="authorino" Aug 04 09:16:26.902853 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.902820 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b2470d3-ff10-4e5d-a3b4-e2f009eb0a8d" containerName="authorino" Aug 04 09:16:26.905775 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.905750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:26.913128 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.912782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:26.913206 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.913171 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-cpvn7"] Aug 04 09:16:26.969067 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.968991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsk8q\" (UniqueName: \"kubernetes.io/projected/92c6677b-1d51-4e32-95f4-d26a6ba43c8e-kube-api-access-rsk8q\") pod \"authorino-5dc7ccb5f-cpvn7\" (UID: \"92c6677b-1d51-4e32-95f4-d26a6ba43c8e\") " pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:26.995402 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:26.995371 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-cpvn7"] Aug 04 09:16:26.995673 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:16:26.995649 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rsk8q], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" podUID="92c6677b-1d51-4e32-95f4-d26a6ba43c8e" Aug 04 09:16:27.024854 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.024825 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-98l75"] Aug 04 09:16:27.028155 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.028139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.030524 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.030502 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Aug 04 09:16:27.036088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.036063 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-98l75"] Aug 04 09:16:27.070330 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.070296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsk8q\" (UniqueName: \"kubernetes.io/projected/92c6677b-1d51-4e32-95f4-d26a6ba43c8e-kube-api-access-rsk8q\") pod \"authorino-5dc7ccb5f-cpvn7\" (UID: \"92c6677b-1d51-4e32-95f4-d26a6ba43c8e\") " pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:27.070448 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.070335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f23a12e-d932-4f40-80be-338667265ac1-tls-cert\") pod \"authorino-687bd45585-98l75\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.070448 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.070381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr9ft\" (UniqueName: \"kubernetes.io/projected/9f23a12e-d932-4f40-80be-338667265ac1-kube-api-access-fr9ft\") pod \"authorino-687bd45585-98l75\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.077966 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.077944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsk8q\" (UniqueName: \"kubernetes.io/projected/92c6677b-1d51-4e32-95f4-d26a6ba43c8e-kube-api-access-rsk8q\") pod \"authorino-5dc7ccb5f-cpvn7\" (UID: \"92c6677b-1d51-4e32-95f4-d26a6ba43c8e\") " pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:27.135962 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.135937 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-98l75"] Aug 04 09:16:27.136199 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:16:27.136179 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fr9ft tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-98l75" podUID="9f23a12e-d932-4f40-80be-338667265ac1" Aug 04 09:16:27.165123 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.165103 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d5649895-9q7vq"] Aug 04 09:16:27.168680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.168661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.171074 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.171043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr9ft\" (UniqueName: \"kubernetes.io/projected/9f23a12e-d932-4f40-80be-338667265ac1-kube-api-access-fr9ft\") pod \"authorino-687bd45585-98l75\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.171177 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.171134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f23a12e-d932-4f40-80be-338667265ac1-tls-cert\") pod \"authorino-687bd45585-98l75\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.173849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.173829 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f23a12e-d932-4f40-80be-338667265ac1-tls-cert\") pod \"authorino-687bd45585-98l75\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.175805 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.175783 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d5649895-9q7vq"] Aug 04 09:16:27.178678 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.178657 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr9ft\" (UniqueName: \"kubernetes.io/projected/9f23a12e-d932-4f40-80be-338667265ac1-kube-api-access-fr9ft\") pod \"authorino-687bd45585-98l75\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.272165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.272107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-tls-cert\") pod \"authorino-7d5649895-9q7vq\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.272165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.272134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dp57\" (UniqueName: \"kubernetes.io/projected/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-kube-api-access-8dp57\") pod \"authorino-7d5649895-9q7vq\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.373538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.373509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-tls-cert\") pod \"authorino-7d5649895-9q7vq\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.373538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.373540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dp57\" (UniqueName: \"kubernetes.io/projected/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-kube-api-access-8dp57\") pod \"authorino-7d5649895-9q7vq\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.375826 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.375807 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-tls-cert\") pod \"authorino-7d5649895-9q7vq\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.382524 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.382501 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dp57\" (UniqueName: \"kubernetes.io/projected/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-kube-api-access-8dp57\") pod \"authorino-7d5649895-9q7vq\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.495129 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.495103 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:16:27.626267 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.626226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d5649895-9q7vq"] Aug 04 09:16:27.627950 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:16:27.627923 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecf30c87_23a0_4e9e_b59d_ff1d8c22e344.slice/crio-4cd825b00e2a2a605ebb5012a210d697e951c9c005ec7dabab7de85f31e7c834 WatchSource:0}: Error finding container 4cd825b00e2a2a605ebb5012a210d697e951c9c005ec7dabab7de85f31e7c834: Status 404 returned error can't find the container with id 4cd825b00e2a2a605ebb5012a210d697e951c9c005ec7dabab7de85f31e7c834 Aug 04 09:16:27.841242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.841155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5649895-9q7vq" event={"ID":"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344","Type":"ContainerStarted","Data":"4cd825b00e2a2a605ebb5012a210d697e951c9c005ec7dabab7de85f31e7c834"} Aug 04 09:16:27.841242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.841207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.841242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.841215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:27.849410 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.849388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:27.851849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.851832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:27.878517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.878494 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f23a12e-d932-4f40-80be-338667265ac1-tls-cert\") pod \"9f23a12e-d932-4f40-80be-338667265ac1\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " Aug 04 09:16:27.878618 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.878549 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr9ft\" (UniqueName: \"kubernetes.io/projected/9f23a12e-d932-4f40-80be-338667265ac1-kube-api-access-fr9ft\") pod \"9f23a12e-d932-4f40-80be-338667265ac1\" (UID: \"9f23a12e-d932-4f40-80be-338667265ac1\") " Aug 04 09:16:27.880526 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.880505 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f23a12e-d932-4f40-80be-338667265ac1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9f23a12e-d932-4f40-80be-338667265ac1" (UID: "9f23a12e-d932-4f40-80be-338667265ac1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:16:27.880612 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.880556 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f23a12e-d932-4f40-80be-338667265ac1-kube-api-access-fr9ft" (OuterVolumeSpecName: "kube-api-access-fr9ft") pod "9f23a12e-d932-4f40-80be-338667265ac1" (UID: "9f23a12e-d932-4f40-80be-338667265ac1"). InnerVolumeSpecName "kube-api-access-fr9ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:16:27.979089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.979064 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsk8q\" (UniqueName: \"kubernetes.io/projected/92c6677b-1d51-4e32-95f4-d26a6ba43c8e-kube-api-access-rsk8q\") pod \"92c6677b-1d51-4e32-95f4-d26a6ba43c8e\" (UID: \"92c6677b-1d51-4e32-95f4-d26a6ba43c8e\") " Aug 04 09:16:27.979238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.979216 2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f23a12e-d932-4f40-80be-338667265ac1-tls-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:16:27.979238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.979226 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr9ft\" (UniqueName: \"kubernetes.io/projected/9f23a12e-d932-4f40-80be-338667265ac1-kube-api-access-fr9ft\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:16:27.980972 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:27.980946 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c6677b-1d51-4e32-95f4-d26a6ba43c8e-kube-api-access-rsk8q" (OuterVolumeSpecName: "kube-api-access-rsk8q") pod "92c6677b-1d51-4e32-95f4-d26a6ba43c8e" (UID: "92c6677b-1d51-4e32-95f4-d26a6ba43c8e"). InnerVolumeSpecName "kube-api-access-rsk8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:16:28.080595 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.080564 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsk8q\" (UniqueName: \"kubernetes.io/projected/92c6677b-1d51-4e32-95f4-d26a6ba43c8e-kube-api-access-rsk8q\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:16:28.853870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.853836 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-cpvn7" Aug 04 09:16:28.853870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.853851 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5649895-9q7vq" event={"ID":"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344","Type":"ContainerStarted","Data":"43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d"} Aug 04 09:16:28.854297 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.854117 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-98l75" Aug 04 09:16:28.870209 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.870169 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d5649895-9q7vq" podStartSLOduration=1.330848893 podStartE2EDuration="1.870154169s" totalImagesPullingTime="539.305276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:16:28.86937307 +0000 UTC m=+506.001582602" watchObservedRunningTime="2026-08-04 09:16:28.870154169 +0000 UTC m=+506.002363700" Aug 04 09:16:28.895497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.895463 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-98l75"] Aug 04 09:16:28.902838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.902811 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-98l75"] Aug 04 09:16:28.904793 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.904774 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qct98"] Aug 04 09:16:28.904974 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.904955 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-qct98" podUID="d808d816-698e-48f4-ab77-32ad46c48847" containerName="authorino" containerID="cri-o://444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42" gracePeriod=30 Aug 04 09:16:28.924049 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.924027 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-cpvn7"] Aug 04 09:16:28.928125 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:28.928104 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-cpvn7"] Aug 04 09:16:29.151951 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.151926 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:16:29.192230 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.192200 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwmwb\" (UniqueName: \"kubernetes.io/projected/d808d816-698e-48f4-ab77-32ad46c48847-kube-api-access-gwmwb\") pod \"d808d816-698e-48f4-ab77-32ad46c48847\" (UID: \"d808d816-698e-48f4-ab77-32ad46c48847\") " Aug 04 09:16:29.194097 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.194074 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d808d816-698e-48f4-ab77-32ad46c48847-kube-api-access-gwmwb" (OuterVolumeSpecName: "kube-api-access-gwmwb") pod "d808d816-698e-48f4-ab77-32ad46c48847" (UID: "d808d816-698e-48f4-ab77-32ad46c48847"). InnerVolumeSpecName "kube-api-access-gwmwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:16:29.293269 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.293228 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwmwb\" (UniqueName: \"kubernetes.io/projected/d808d816-698e-48f4-ab77-32ad46c48847-kube-api-access-gwmwb\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:16:29.416310 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.416278 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c6677b-1d51-4e32-95f4-d26a6ba43c8e" path="/var/lib/kubelet/pods/92c6677b-1d51-4e32-95f4-d26a6ba43c8e/volumes" Aug 04 09:16:29.416557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.416545 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f23a12e-d932-4f40-80be-338667265ac1" path="/var/lib/kubelet/pods/9f23a12e-d932-4f40-80be-338667265ac1/volumes" Aug 04 09:16:29.860029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.859935 2562 generic.go:356] "Generic (PLEG): container finished" podID="d808d816-698e-48f4-ab77-32ad46c48847" containerID="444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42" exitCode=0 Aug 04 09:16:29.860029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.859983 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-qct98" Aug 04 09:16:29.860558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.860022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qct98" event={"ID":"d808d816-698e-48f4-ab77-32ad46c48847","Type":"ContainerDied","Data":"444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42"} Aug 04 09:16:29.860558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.860067 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-qct98" event={"ID":"d808d816-698e-48f4-ab77-32ad46c48847","Type":"ContainerDied","Data":"ec5ea4d7bcf8e78edc6b2a5c33b027fa08bd6f99b7b2bd78269fafcc2374e561"} Aug 04 09:16:29.860558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.860087 2562 scope.go:117] "RemoveContainer" containerID="444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42" Aug 04 09:16:29.868785 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.868770 2562 scope.go:117] "RemoveContainer" containerID="444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42" Aug 04 09:16:29.869016 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:16:29.868999 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42\": container with ID starting with 444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42 not found: ID does not exist" containerID="444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42" Aug 04 09:16:29.869059 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.869031 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42"} err="failed to get container status \"444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42\": rpc error: code = NotFound desc = could not find container \"444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42\": container with ID starting with 444bf40e251b77ae87297b42dd7f673028904eb857904b480350c606d60e6a42 not found: ID does not exist" Aug 04 09:16:29.876682 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.876660 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qct98"] Aug 04 09:16:29.880351 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:29.880330 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-qct98"] Aug 04 09:16:31.415688 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:31.415657 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d808d816-698e-48f4-ab77-32ad46c48847" path="/var/lib/kubelet/pods/d808d816-698e-48f4-ab77-32ad46c48847/volumes" Aug 04 09:16:33.468551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:33.468517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:33.543172 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:33.543143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:33.607578 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:33.607525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:36.468835 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:36.468803 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:36.568653 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:36.568617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:16:48.637596 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:16:48.637559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:10.578580 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:10.578544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:11.998567 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:11.998529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:12.098178 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.098134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:12.511322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.511287 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:12.622635 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.622591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:12.712705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.712671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:12.777286 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.777172 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r"] Aug 04 09:17:12.777695 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.777667 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d808d816-698e-48f4-ab77-32ad46c48847" containerName="authorino" Aug 04 09:17:12.777791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.777698 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d808d816-698e-48f4-ab77-32ad46c48847" containerName="authorino" Aug 04 09:17:12.777791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.777787 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d808d816-698e-48f4-ab77-32ad46c48847" containerName="authorino" Aug 04 09:17:12.782282 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.782234 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:12.785137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.785114 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bxljn\"" Aug 04 09:17:12.785137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.785128 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Aug 04 09:17:12.792507 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.792483 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r"] Aug 04 09:17:12.885950 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.885907 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d43c1705-8f57-412d-bd66-cf9b141ece3b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-ncr4r\" (UID: \"d43c1705-8f57-412d-bd66-cf9b141ece3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:12.886134 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.886021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7pc8\" (UniqueName: \"kubernetes.io/projected/d43c1705-8f57-412d-bd66-cf9b141ece3b-kube-api-access-x7pc8\") pod \"payload-pre-processing-687dbb6cf6-ncr4r\" (UID: \"d43c1705-8f57-412d-bd66-cf9b141ece3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:12.986906 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.986874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7pc8\" (UniqueName: \"kubernetes.io/projected/d43c1705-8f57-412d-bd66-cf9b141ece3b-kube-api-access-x7pc8\") pod \"payload-pre-processing-687dbb6cf6-ncr4r\" (UID: \"d43c1705-8f57-412d-bd66-cf9b141ece3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:12.986906 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.986912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d43c1705-8f57-412d-bd66-cf9b141ece3b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-ncr4r\" (UID: \"d43c1705-8f57-412d-bd66-cf9b141ece3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:12.987615 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.987595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d43c1705-8f57-412d-bd66-cf9b141ece3b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-ncr4r\" (UID: \"d43c1705-8f57-412d-bd66-cf9b141ece3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:12.999289 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:12.999243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7pc8\" (UniqueName: \"kubernetes.io/projected/d43c1705-8f57-412d-bd66-cf9b141ece3b-kube-api-access-x7pc8\") pod \"payload-pre-processing-687dbb6cf6-ncr4r\" (UID: \"d43c1705-8f57-412d-bd66-cf9b141ece3b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:13.092954 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:13.092872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:13.220900 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:13.220874 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r"] Aug 04 09:17:13.222998 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:17:13.222968 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43c1705_8f57_412d_bd66_cf9b141ece3b.slice/crio-dc861129250d17872ae84edac335a202c63463998d0264343ac550b9eaedfd0f WatchSource:0}: Error finding container dc861129250d17872ae84edac335a202c63463998d0264343ac550b9eaedfd0f: Status 404 returned error can't find the container with id dc861129250d17872ae84edac335a202c63463998d0264343ac550b9eaedfd0f Aug 04 09:17:14.091865 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:14.091825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" event={"ID":"d43c1705-8f57-412d-bd66-cf9b141ece3b","Type":"ContainerStarted","Data":"dc861129250d17872ae84edac335a202c63463998d0264343ac550b9eaedfd0f"} Aug 04 09:17:15.097909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:15.097871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" event={"ID":"d43c1705-8f57-412d-bd66-cf9b141ece3b","Type":"ContainerStarted","Data":"66d6b273048d28920b05ca29852d62e75c8687bca9b71900be55bcc81d216b3c"} Aug 04 09:17:15.098310 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:15.097931 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:15.113795 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:15.113744 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" podStartSLOduration=1.4386733839999999 podStartE2EDuration="3.113729248s" totalImagesPullingTime="1.675055864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:17:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:17:15.112833644 +0000 UTC m=+552.245043178" watchObservedRunningTime="2026-08-04 09:17:15.113729248 +0000 UTC m=+552.245938837" Aug 04 09:17:22.259130 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:22.259091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:22.356637 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:22.356595 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:26.104811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:26.104774 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ncr4r" Aug 04 09:17:34.237239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:34.237202 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:34.337630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:34.337598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:36.360090 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:36.360056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:49.766833 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:49.766793 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:17:59.258911 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:17:59.258876 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:18:09.338335 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.338300 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5"] Aug 04 09:18:09.341785 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.341767 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.345481 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.345454 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 09:18:09.345613 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.345454 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pfrgq\"" Aug 04 09:18:09.345613 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.345495 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 09:18:09.345613 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.345528 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Aug 04 09:18:09.353852 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.353834 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5"] Aug 04 09:18:09.374456 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.374432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:18:09.480522 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.480495 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.480674 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.480551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.480736 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.480711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.480795 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.480779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddt2j\" (UniqueName: \"kubernetes.io/projected/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-kube-api-access-ddt2j\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.480943 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.480923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.480999 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.480951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.582514 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.582477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddt2j\" (UniqueName: \"kubernetes.io/projected/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-kube-api-access-ddt2j\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.582701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.582653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.582701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.582686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.582836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.582756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.582836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.582785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.583089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.582961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.583335 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.583308 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.583455 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.583341 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.583455 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.583400 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.585338 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.585311 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.585530 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.585512 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.591850 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.591786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddt2j\" (UniqueName: \"kubernetes.io/projected/c33ac7ab-cd4d-4844-891d-f0b80687f3d0-kube-api-access-ddt2j\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5\" (UID: \"c33ac7ab-cd4d-4844-891d-f0b80687f3d0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.653027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.653001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:09.784041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.784011 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5"] Aug 04 09:18:09.785745 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:18:09.785713 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc33ac7ab_cd4d_4844_891d_f0b80687f3d0.slice/crio-4b4780c9989d01eb5e88df4cfb43c31410a8f83eecdc127ae471e7eee934559d WatchSource:0}: Error finding container 4b4780c9989d01eb5e88df4cfb43c31410a8f83eecdc127ae471e7eee934559d: Status 404 returned error can't find the container with id 4b4780c9989d01eb5e88df4cfb43c31410a8f83eecdc127ae471e7eee934559d Aug 04 09:18:09.787100 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:09.787082 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:18:10.385132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:10.385088 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" event={"ID":"c33ac7ab-cd4d-4844-891d-f0b80687f3d0","Type":"ContainerStarted","Data":"4b4780c9989d01eb5e88df4cfb43c31410a8f83eecdc127ae471e7eee934559d"} Aug 04 09:18:11.391740 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:11.391699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" event={"ID":"c33ac7ab-cd4d-4844-891d-f0b80687f3d0","Type":"ContainerStarted","Data":"757fba2ed01827d6f7bf89ebace6202f26d0cf97272272e8dc75d113a1fb1da7"} Aug 04 09:18:11.409450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:11.409397 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" podStartSLOduration=1.360922625 podStartE2EDuration="2.409382901s" totalImagesPullingTime="1.048460276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:18:11.408123455 +0000 UTC m=+608.540332987" watchObservedRunningTime="2026-08-04 09:18:11.409382901 +0000 UTC m=+608.541592433" Aug 04 09:18:19.653824 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:19.653789 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:19.654284 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:19.654075 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:19.666306 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:19.666286 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:20.448676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:20.448644 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5" Aug 04 09:18:24.338119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:24.338086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:18:29.568198 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:29.568117 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:18:51.219807 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:51.219772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:18:51.282828 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:18:51.282794 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:16.252460 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:16.252420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:16.352447 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:16.352395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:16.763818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:16.763789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:23.652026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.651984 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c4678dcd-f8s65"] Aug 04 09:19:23.655634 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.655610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.663304 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.663275 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c4678dcd-f8s65"] Aug 04 09:19:23.752041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.752013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gtfv\" (UniqueName: \"kubernetes.io/projected/a42398c4-7076-49d1-aec3-bd0a03add1e6-kube-api-access-4gtfv\") pod \"authorino-56c4678dcd-f8s65\" (UID: \"a42398c4-7076-49d1-aec3-bd0a03add1e6\") " pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.752170 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.752052 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a42398c4-7076-49d1-aec3-bd0a03add1e6-tls-cert\") pod \"authorino-56c4678dcd-f8s65\" (UID: \"a42398c4-7076-49d1-aec3-bd0a03add1e6\") " pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.852866 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.852834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gtfv\" (UniqueName: \"kubernetes.io/projected/a42398c4-7076-49d1-aec3-bd0a03add1e6-kube-api-access-4gtfv\") pod \"authorino-56c4678dcd-f8s65\" (UID: \"a42398c4-7076-49d1-aec3-bd0a03add1e6\") " pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.853001 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.852878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a42398c4-7076-49d1-aec3-bd0a03add1e6-tls-cert\") pod \"authorino-56c4678dcd-f8s65\" (UID: \"a42398c4-7076-49d1-aec3-bd0a03add1e6\") " pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.855222 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.855193 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a42398c4-7076-49d1-aec3-bd0a03add1e6-tls-cert\") pod \"authorino-56c4678dcd-f8s65\" (UID: \"a42398c4-7076-49d1-aec3-bd0a03add1e6\") " pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.861990 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.861966 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gtfv\" (UniqueName: \"kubernetes.io/projected/a42398c4-7076-49d1-aec3-bd0a03add1e6-kube-api-access-4gtfv\") pod \"authorino-56c4678dcd-f8s65\" (UID: \"a42398c4-7076-49d1-aec3-bd0a03add1e6\") " pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:23.965841 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:23.965772 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c4678dcd-f8s65" Aug 04 09:19:24.091307 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:24.091285 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c4678dcd-f8s65"] Aug 04 09:19:24.093079 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:19:24.093044 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda42398c4_7076_49d1_aec3_bd0a03add1e6.slice/crio-d9a245c152340faa3a01d467e1d753d42261d068ac78fc1b1157c01edacbb7ad WatchSource:0}: Error finding container d9a245c152340faa3a01d467e1d753d42261d068ac78fc1b1157c01edacbb7ad: Status 404 returned error can't find the container with id d9a245c152340faa3a01d467e1d753d42261d068ac78fc1b1157c01edacbb7ad Aug 04 09:19:24.785230 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:24.785194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c4678dcd-f8s65" event={"ID":"a42398c4-7076-49d1-aec3-bd0a03add1e6","Type":"ContainerStarted","Data":"d9a245c152340faa3a01d467e1d753d42261d068ac78fc1b1157c01edacbb7ad"} Aug 04 09:19:25.793005 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:25.792958 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c4678dcd-f8s65" event={"ID":"a42398c4-7076-49d1-aec3-bd0a03add1e6","Type":"ContainerStarted","Data":"dbcd18c460885921bc963f4f0872bf3d81e3eccc96284917fa785b2315dcd453"} Aug 04 09:19:25.816351 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:25.816281 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56c4678dcd-f8s65" podStartSLOduration=1.933917093 podStartE2EDuration="2.816240087s" totalImagesPullingTime="882.322994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:19:25.813408168 +0000 UTC m=+682.945617702" watchObservedRunningTime="2026-08-04 09:19:25.816240087 +0000 UTC m=+682.948449620" Aug 04 09:19:25.853401 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:25.853361 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d5649895-9q7vq"] Aug 04 09:19:25.853578 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:25.853555 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d5649895-9q7vq" podUID="ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" containerName="authorino" containerID="cri-o://43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d" gracePeriod=30 Aug 04 09:19:26.103743 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.103710 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:19:26.176550 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.175555 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dp57\" (UniqueName: \"kubernetes.io/projected/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-kube-api-access-8dp57\") pod \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " Aug 04 09:19:26.176550 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.175688 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-tls-cert\") pod \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\" (UID: \"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344\") " Aug 04 09:19:26.184089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.184039 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-kube-api-access-8dp57" (OuterVolumeSpecName: "kube-api-access-8dp57") pod "ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" (UID: "ecf30c87-23a0-4e9e-b59d-ff1d8c22e344"). InnerVolumeSpecName "kube-api-access-8dp57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:19:26.196537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.196507 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" (UID: "ecf30c87-23a0-4e9e-b59d-ff1d8c22e344"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:19:26.277428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.277376 2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-tls-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:19:26.277428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.277428 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dp57\" (UniqueName: \"kubernetes.io/projected/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344-kube-api-access-8dp57\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:19:26.396523 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.396490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:26.492652 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.492615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:26.800633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.800541 2562 generic.go:356] "Generic (PLEG): container finished" podID="ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" containerID="43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d" exitCode=0 Aug 04 09:19:26.800633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.800584 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5649895-9q7vq" Aug 04 09:19:26.800633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.800625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5649895-9q7vq" event={"ID":"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344","Type":"ContainerDied","Data":"43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d"} Aug 04 09:19:26.801180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.800670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5649895-9q7vq" event={"ID":"ecf30c87-23a0-4e9e-b59d-ff1d8c22e344","Type":"ContainerDied","Data":"4cd825b00e2a2a605ebb5012a210d697e951c9c005ec7dabab7de85f31e7c834"} Aug 04 09:19:26.801180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.800692 2562 scope.go:117] "RemoveContainer" containerID="43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d" Aug 04 09:19:26.810142 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.810124 2562 scope.go:117] "RemoveContainer" containerID="43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d" Aug 04 09:19:26.810467 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:19:26.810448 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d\": container with ID starting with 43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d not found: ID does not exist" containerID="43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d" Aug 04 09:19:26.810542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.810476 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d"} err="failed to get container status \"43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d\": rpc error: code = NotFound desc = could not find container \"43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d\": container with ID starting with 43429c24a89244ab1c96d3c7b94176088c614224b4bcc2be7e354a178eeb682d not found: ID does not exist" Aug 04 09:19:26.842836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.842805 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d5649895-9q7vq"] Aug 04 09:19:26.858187 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:26.858161 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d5649895-9q7vq"] Aug 04 09:19:27.416909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:27.416866 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" path="/var/lib/kubelet/pods/ecf30c87-23a0-4e9e-b59d-ff1d8c22e344/volumes" Aug 04 09:19:27.614699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:27.614664 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:19:27.749956 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:19:27.749872 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:20:25.672789 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:20:25.672753 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:20:40.475144 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:20:40.475107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:21:34.185199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:21:34.185168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:21:35.446206 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:21:35.446167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:38.575286 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:38.575230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:38.671026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:38.670991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:38.803366 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:38.803331 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:44.667480 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:44.667444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:44.800218 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:44.800183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:44.891981 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:44.891946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:45.097219 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:45.097139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:50.831675 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:50.831636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:22:50.931079 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:22:50.931045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:25:24.432612 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:25:24.432574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:28:30.637210 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:28:30.637178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:30:00.138400 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.138359 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws"] Aug 04 09:30:00.146434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.138906 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" containerName="authorino" Aug 04 09:30:00.146434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.138923 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" containerName="authorino" Aug 04 09:30:00.146434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.138993 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf30c87-23a0-4e9e-b59d-ff1d8c22e344" containerName="authorino" Aug 04 09:30:00.146434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.146428 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" Aug 04 09:30:00.149273 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.149219 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:30:00.150815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.150787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws"] Aug 04 09:30:00.158621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.158601 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" Aug 04 09:30:00.288345 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.288320 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws"] Aug 04 09:30:00.290384 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:30:00.290359 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e889f0_d4c9_47b4_924e_a39c03e9ea6c.slice/crio-ee7f097baeb25170ee3f9b57763a46cc07bf0b913cf32db0c236f0d1a68cc2ac WatchSource:0}: Error finding container ee7f097baeb25170ee3f9b57763a46cc07bf0b913cf32db0c236f0d1a68cc2ac: Status 404 returned error can't find the container with id ee7f097baeb25170ee3f9b57763a46cc07bf0b913cf32db0c236f0d1a68cc2ac Aug 04 09:30:00.292294 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:00.292275 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:30:01.120476 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:01.120442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" event={"ID":"e0e889f0-d4c9-47b4-924e-a39c03e9ea6c","Type":"ContainerStarted","Data":"ee7f097baeb25170ee3f9b57763a46cc07bf0b913cf32db0c236f0d1a68cc2ac"} Aug 04 09:30:03.132547 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:03.132459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" event={"ID":"e0e889f0-d4c9-47b4-924e-a39c03e9ea6c","Type":"ContainerStarted","Data":"22d85ac0b90a902ad36ad2309e0e7c3e2c0d09ac02671bc4bc244009c122155a"} Aug 04 09:30:03.149424 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:03.149380 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" podStartSLOduration=0.65459524 podStartE2EDuration="3.149365628s" totalImagesPullingTime="2.494770388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:30:03.148279946 +0000 UTC m=+1320.280489475" watchObservedRunningTime="2026-08-04 09:30:03.149365628 +0000 UTC m=+1320.281575160" Aug 04 09:30:04.139024 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:04.138937 2562 generic.go:356] "Generic (PLEG): container finished" podID="e0e889f0-d4c9-47b4-924e-a39c03e9ea6c" containerID="22d85ac0b90a902ad36ad2309e0e7c3e2c0d09ac02671bc4bc244009c122155a" exitCode=0 Aug 04 09:30:04.139386 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:04.139025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" event={"ID":"e0e889f0-d4c9-47b4-924e-a39c03e9ea6c","Type":"ContainerDied","Data":"22d85ac0b90a902ad36ad2309e0e7c3e2c0d09ac02671bc4bc244009c122155a"} Aug 04 09:30:05.278670 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:05.278644 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" Aug 04 09:30:06.150759 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:06.150729 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" Aug 04 09:30:06.150923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:06.150761 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763930-2wdws" event={"ID":"e0e889f0-d4c9-47b4-924e-a39c03e9ea6c","Type":"ContainerDied","Data":"ee7f097baeb25170ee3f9b57763a46cc07bf0b913cf32db0c236f0d1a68cc2ac"} Aug 04 09:30:06.150923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:06.150810 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee7f097baeb25170ee3f9b57763a46cc07bf0b913cf32db0c236f0d1a68cc2ac" Aug 04 09:30:17.232545 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:17.232508 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:30:37.540893 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:30:37.540835 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:32:11.085416 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:11.085332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:32:15.385080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:15.385037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:32:31.747009 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.746973 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh"] Aug 04 09:32:31.747588 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.747567 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e889f0-d4c9-47b4-924e-a39c03e9ea6c" containerName="cleanup" Aug 04 09:32:31.747633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.747591 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e889f0-d4c9-47b4-924e-a39c03e9ea6c" containerName="cleanup" Aug 04 09:32:31.747724 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.747712 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e889f0-d4c9-47b4-924e-a39c03e9ea6c" containerName="cleanup" Aug 04 09:32:31.751111 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.751089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.759700 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.759669 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-417cca6a-kserve-self-signed-certs\"" Aug 04 09:32:31.779186 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.779156 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh"] Aug 04 09:32:31.812461 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.812436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-dshm\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.812581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.812468 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tls-certs\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.812581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.812494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-home\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.812581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.812560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr2rt\" (UniqueName: \"kubernetes.io/projected/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-kube-api-access-lr2rt\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.812714 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.812598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-model-cache\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.812714 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.812632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tmp-dir\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.913813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.913778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tmp-dir\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.913977 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.913824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-dshm\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.913977 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.913912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tls-certs\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.913977 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.913939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-home\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.913977 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.913969 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr2rt\" (UniqueName: \"kubernetes.io/projected/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-kube-api-access-lr2rt\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.914196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.914004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-model-cache\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.914196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.914177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tmp-dir\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.914330 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.914224 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-home\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.914384 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.914368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-model-cache\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.916198 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.916174 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-dshm\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.916351 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.916313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tls-certs\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:31.924385 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:31.924361 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr2rt\" (UniqueName: \"kubernetes.io/projected/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-kube-api-access-lr2rt\") pod \"e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:32.060908 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:32.060825 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:32.204715 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:32.204525 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh"] Aug 04 09:32:32.208225 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:32:32.208196 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b9c56e9_21ae_4b2d_bcee_c46f29f7d581.slice/crio-eb1fc388d12020de15e86a3c5c7e9ac31e9038c37515e71afa6f71cabb1b99e8 WatchSource:0}: Error finding container eb1fc388d12020de15e86a3c5c7e9ac31e9038c37515e71afa6f71cabb1b99e8: Status 404 returned error can't find the container with id eb1fc388d12020de15e86a3c5c7e9ac31e9038c37515e71afa6f71cabb1b99e8 Aug 04 09:32:32.930960 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:32.930918 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" event={"ID":"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581","Type":"ContainerStarted","Data":"0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81"} Aug 04 09:32:32.931352 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:32.930968 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" event={"ID":"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581","Type":"ContainerStarted","Data":"eb1fc388d12020de15e86a3c5c7e9ac31e9038c37515e71afa6f71cabb1b99e8"} Aug 04 09:32:32.958900 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:32.958844 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" podStartSLOduration=1.9588273680000001 podStartE2EDuration="1.958827368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:32:32.958312364 +0000 UTC m=+1470.090521898" watchObservedRunningTime="2026-08-04 09:32:32.958827368 +0000 UTC m=+1470.091036900" Aug 04 09:32:33.414240 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:33.414210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:32:33.512928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:33.512897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:32:42.061134 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:42.061096 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:42.061667 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:42.061160 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:42.073213 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:42.073185 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:42.998309 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:42.998283 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:32:48.902551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:48.902519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:32:49.828178 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:49.828142 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh"] Aug 04 09:32:49.851071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:32:49.851042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:33:04.847201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:04.847165 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" podUID="8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" containerName="main" containerID="cri-o://0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81" gracePeriod=45 Aug 04 09:33:05.097502 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.097444 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:33:05.107620 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.107593 2562 generic.go:356] "Generic (PLEG): container finished" podID="8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" containerID="0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81" exitCode=0 Aug 04 09:33:05.107755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.107639 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" Aug 04 09:33:05.107755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.107672 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" event={"ID":"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581","Type":"ContainerDied","Data":"0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81"} Aug 04 09:33:05.107755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.107710 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh" event={"ID":"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581","Type":"ContainerDied","Data":"eb1fc388d12020de15e86a3c5c7e9ac31e9038c37515e71afa6f71cabb1b99e8"} Aug 04 09:33:05.107755 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.107729 2562 scope.go:117] "RemoveContainer" containerID="0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81" Aug 04 09:33:05.117177 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.117161 2562 scope.go:117] "RemoveContainer" containerID="0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81" Aug 04 09:33:05.117475 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:33:05.117455 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81\": container with ID starting with 0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81 not found: ID does not exist" containerID="0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81" Aug 04 09:33:05.117572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.117482 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81"} err="failed to get container status \"0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81\": rpc error: code = NotFound desc = could not find container \"0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81\": container with ID starting with 0978eb44861d8fce8b1951c0c3f9f50936640673b875089a3bd3f616de5ebf81 not found: ID does not exist" Aug 04 09:33:05.217397 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217366 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-home\") pod \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " Aug 04 09:33:05.217543 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217413 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tmp-dir\") pod \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " Aug 04 09:33:05.217543 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217438 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-dshm\") pod \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " Aug 04 09:33:05.217543 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217493 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-model-cache\") pod \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " Aug 04 09:33:05.217693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217543 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tls-certs\") pod \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " Aug 04 09:33:05.217693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217568 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr2rt\" (UniqueName: \"kubernetes.io/projected/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-kube-api-access-lr2rt\") pod \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\" (UID: \"8b9c56e9-21ae-4b2d-bcee-c46f29f7d581\") " Aug 04 09:33:05.217693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217616 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-home" (OuterVolumeSpecName: "home") pod "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" (UID: "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:33:05.217693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217631 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" (UID: "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:33:05.217905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217828 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" (UID: "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:33:05.217960 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217930 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-home\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:33:05.217960 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217949 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tmp-dir\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:33:05.218037 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.217965 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-model-cache\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:33:05.219633 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.219608 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-dshm" (OuterVolumeSpecName: "dshm") pod "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" (UID: "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:33:05.219744 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.219669 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-kube-api-access-lr2rt" (OuterVolumeSpecName: "kube-api-access-lr2rt") pod "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" (UID: "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581"). InnerVolumeSpecName "kube-api-access-lr2rt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:33:05.219744 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.219682 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" (UID: "8b9c56e9-21ae-4b2d-bcee-c46f29f7d581"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:33:05.318859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.318824 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-tls-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:33:05.318859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.318853 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr2rt\" (UniqueName: \"kubernetes.io/projected/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-kube-api-access-lr2rt\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:33:05.318859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.318863 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581-dshm\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:33:05.432949 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.432916 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh"] Aug 04 09:33:05.437648 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:05.437622 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-417cca6a-kserve-569bc7f5f5-jkskh"] Aug 04 09:33:07.418179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:33:07.418123 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" path="/var/lib/kubelet/pods/8b9c56e9-21ae-4b2d-bcee-c46f29f7d581/volumes" Aug 04 09:36:10.896364 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:10.896325 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:37.365196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:37.365159 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:41.002503 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:41.002460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:41.095044 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:41.095006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:41.557639 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:41.557603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:47.462427 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:47.462394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:47.531731 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:47.531703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:47.738461 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:47.738354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:48.107807 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.107730 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg"] Aug 04 09:36:48.108106 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.108094 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" containerName="main" Aug 04 09:36:48.108149 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.108108 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" containerName="main" Aug 04 09:36:48.108183 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.108168 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9c56e9-21ae-4b2d-bcee-c46f29f7d581" containerName="main" Aug 04 09:36:48.111261 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.111231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.113678 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.113661 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e842246f-openshift-default-dockercfg-jjcpw\"" Aug 04 09:36:48.137046 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137016 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded2346f-5281-42c4-8e92-647043657be2-istio-podinfo\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-credential-socket\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137151 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137132 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded2346f-5281-42c4-8e92-647043657be2-istiod-ca-cert\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-data\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-istio-token\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-socket\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-envoy\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137302 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zfrx\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-kube-api-access-8zfrx\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.137474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.137329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-certs\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.138560 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.138542 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg"] Aug 04 09:36:48.142102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.142073 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:48.238327 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded2346f-5281-42c4-8e92-647043657be2-istio-podinfo\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-credential-socket\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded2346f-5281-42c4-8e92-647043657be2-istiod-ca-cert\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-data\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-istio-token\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238712 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-socket\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238712 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-envoy\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238712 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zfrx\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-kube-api-access-8zfrx\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-certs\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-credential-socket\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.238874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238862 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-data\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.239050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.238947 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-socket\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.239109 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.239054 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-certs\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.239428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.239403 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded2346f-5281-42c4-8e92-647043657be2-istiod-ca-cert\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.240674 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.240653 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-envoy\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.240804 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.240788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded2346f-5281-42c4-8e92-647043657be2-istio-podinfo\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.247264 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.247232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-istio-token\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.247420 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.247401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zfrx\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-kube-api-access-8zfrx\") pod \"e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.331503 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.331466 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:48.422378 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.422345 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:48.559765 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.559740 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg"] Aug 04 09:36:48.561408 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:36:48.561379 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded2346f_5281_42c4_8e92_647043657be2.slice/crio-be0090582767238f368b76c7c76f6028ad09d88f267c0a157fa4c358424e8c1b WatchSource:0}: Error finding container be0090582767238f368b76c7c76f6028ad09d88f267c0a157fa4c358424e8c1b: Status 404 returned error can't find the container with id be0090582767238f368b76c7c76f6028ad09d88f267c0a157fa4c358424e8c1b Aug 04 09:36:48.563650 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.563621 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:36:48.563725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.563692 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:36:48.563725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:48.563719 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:36:49.333227 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:49.333186 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" event={"ID":"ded2346f-5281-42c4-8e92-647043657be2","Type":"ContainerStarted","Data":"675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084"} Aug 04 09:36:49.333449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:49.333241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" event={"ID":"ded2346f-5281-42c4-8e92-647043657be2","Type":"ContainerStarted","Data":"be0090582767238f368b76c7c76f6028ad09d88f267c0a157fa4c358424e8c1b"} Aug 04 09:36:49.358061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:49.358008 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podStartSLOduration=1.357990683 podStartE2EDuration="1.357990683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:36:49.355027834 +0000 UTC m=+1726.487237368" watchObservedRunningTime="2026-08-04 09:36:49.357990683 +0000 UTC m=+1726.490200215" Aug 04 09:36:49.423024 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:49.422993 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:49.424542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:49.424520 2562 patch_prober.go:28] interesting pod/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Aug 04 09:36:49.424597 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:49.424564 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Aug 04 09:36:50.427187 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:50.427157 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:50.738406 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:50.738280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:50.829264 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:50.829232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:51.344163 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:51.344124 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:51.344983 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:51.344959 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:36:51.863688 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:51.863658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:52.494820 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.494783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:52.602536 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.602505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:36:52.702910 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.702881 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h"] Aug 04 09:36:52.706327 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.706305 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:52.709073 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.709052 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e842246f\"" Aug 04 09:36:52.709161 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.709133 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e842246f-dockercfg-c6twr\"" Aug 04 09:36:52.719014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.718984 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h"] Aug 04 09:36:52.783317 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.783221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksltk\" (UniqueName: \"kubernetes.io/projected/438c5991-0220-4fa1-bdd8-1d2464573729-kube-api-access-ksltk\") pod \"payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:52.783483 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.783367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438c5991-0220-4fa1-bdd8-1d2464573729-plugins-config-volume\") pod \"payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:52.884713 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.884678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438c5991-0220-4fa1-bdd8-1d2464573729-plugins-config-volume\") pod \"payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:52.885185 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.884765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksltk\" (UniqueName: \"kubernetes.io/projected/438c5991-0220-4fa1-bdd8-1d2464573729-kube-api-access-ksltk\") pod \"payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:52.885560 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.885532 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438c5991-0220-4fa1-bdd8-1d2464573729-plugins-config-volume\") pod \"payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:52.893261 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:52.893227 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksltk\" (UniqueName: \"kubernetes.io/projected/438c5991-0220-4fa1-bdd8-1d2464573729-kube-api-access-ksltk\") pod \"payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:53.017951 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:53.017919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:36:53.152939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:53.152909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h"] Aug 04 09:36:53.155125 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:36:53.155097 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438c5991_0220_4fa1_bdd8_1d2464573729.slice/crio-ba12dab68f86841898ba13d4e83f4cde4c7454795e8e6ed8c5738d35336844e4 WatchSource:0}: Error finding container ba12dab68f86841898ba13d4e83f4cde4c7454795e8e6ed8c5738d35336844e4: Status 404 returned error can't find the container with id ba12dab68f86841898ba13d4e83f4cde4c7454795e8e6ed8c5738d35336844e4 Aug 04 09:36:53.356434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:53.356354 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" event={"ID":"438c5991-0220-4fa1-bdd8-1d2464573729","Type":"ContainerStarted","Data":"5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0"} Aug 04 09:36:53.356434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:53.356415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" event={"ID":"438c5991-0220-4fa1-bdd8-1d2464573729","Type":"ContainerStarted","Data":"ba12dab68f86841898ba13d4e83f4cde4c7454795e8e6ed8c5738d35336844e4"} Aug 04 09:36:53.375883 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:53.375831 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" podStartSLOduration=1.375810358 podStartE2EDuration="1.375810358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:36:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:36:53.37382148 +0000 UTC m=+1730.506031012" watchObservedRunningTime="2026-08-04 09:36:53.375810358 +0000 UTC m=+1730.508019894" Aug 04 09:36:54.362268 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:36:54.362221 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:37:04.834318 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:37:04.834291 2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded2346f_5281_42c4_8e92_647043657be2.slice/crio-675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084.scope\": RecentStats: unable to find data in memory cache]" Aug 04 09:37:05.370682 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.370646 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:37:05.425349 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.425319 2562 generic.go:356] "Generic (PLEG): container finished" podID="ded2346f-5281-42c4-8e92-647043657be2" containerID="675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084" exitCode=0 Aug 04 09:37:05.425506 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.425390 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" event={"ID":"ded2346f-5281-42c4-8e92-647043657be2","Type":"ContainerDied","Data":"675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084"} Aug 04 09:37:05.425838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.425823 2562 scope.go:117] "RemoveContainer" containerID="675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084" Aug 04 09:37:05.426913 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.426891 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:05.426972 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.426953 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:05.427008 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:05.426990 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:06.344964 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:06.344917 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:06.432777 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:06.432740 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" event={"ID":"ded2346f-5281-42c4-8e92-647043657be2","Type":"ContainerStarted","Data":"56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6"} Aug 04 09:37:07.422558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:07.422511 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:07.424111 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:07.424081 2562 patch_prober.go:28] interesting pod/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Aug 04 09:37:07.424259 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:07.424134 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Aug 04 09:37:08.423735 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:08.423699 2562 patch_prober.go:28] interesting pod/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Aug 04 09:37:08.424132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:08.423753 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Aug 04 09:37:09.422881 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:09.422844 2562 patch_prober.go:28] interesting pod/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Aug 04 09:37:09.423055 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:09.422911 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Aug 04 09:37:10.423066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:10.423029 2562 patch_prober.go:28] interesting pod/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Aug 04 09:37:10.423474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:10.423097 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Aug 04 09:37:11.426817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:11.426787 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:11.427200 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:11.427043 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:11.427817 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:11.427798 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:21.701584 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.701553 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9"] Aug 04 09:37:21.705106 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.705089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:21.707621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.707597 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:37:21.707747 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.707625 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 09:37:21.713467 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.712911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9"] Aug 04 09:37:21.754734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.754703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec1cb1b-2a63-4f63-bf39-882ad709e15b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e842246f-5f7f9\" (UID: \"5ec1cb1b-2a63-4f63-bf39-882ad709e15b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:21.855771 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.855735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec1cb1b-2a63-4f63-bf39-882ad709e15b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e842246f-5f7f9\" (UID: \"5ec1cb1b-2a63-4f63-bf39-882ad709e15b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:21.856349 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:21.856329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec1cb1b-2a63-4f63-bf39-882ad709e15b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e842246f-5f7f9\" (UID: \"5ec1cb1b-2a63-4f63-bf39-882ad709e15b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:22.016827 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:22.016744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:22.350849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:22.350823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9"] Aug 04 09:37:22.352579 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:37:22.352548 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec1cb1b_2a63_4f63_bf39_882ad709e15b.slice/crio-1cd5d92ae455da03bc8415e496ec3fab80678a04872608ce638ca2db9c50c0f0 WatchSource:0}: Error finding container 1cd5d92ae455da03bc8415e496ec3fab80678a04872608ce638ca2db9c50c0f0: Status 404 returned error can't find the container with id 1cd5d92ae455da03bc8415e496ec3fab80678a04872608ce638ca2db9c50c0f0 Aug 04 09:37:22.533797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:22.533756 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" event={"ID":"5ec1cb1b-2a63-4f63-bf39-882ad709e15b","Type":"ContainerStarted","Data":"1cd5d92ae455da03bc8415e496ec3fab80678a04872608ce638ca2db9c50c0f0"} Aug 04 09:37:23.540457 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:23.540420 2562 generic.go:356] "Generic (PLEG): container finished" podID="5ec1cb1b-2a63-4f63-bf39-882ad709e15b" containerID="12be4cea2a24ab0dc1a58d24c0667bfea243f46b51b6693ca19a7c0cbf7e89d7" exitCode=0 Aug 04 09:37:23.540821 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:23.540497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" event={"ID":"5ec1cb1b-2a63-4f63-bf39-882ad709e15b","Type":"ContainerDied","Data":"12be4cea2a24ab0dc1a58d24c0667bfea243f46b51b6693ca19a7c0cbf7e89d7"} Aug 04 09:37:24.674398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:24.674370 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:24.676558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:24.676538 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec1cb1b-2a63-4f63-bf39-882ad709e15b-maas-api-service-ca\") pod \"5ec1cb1b-2a63-4f63-bf39-882ad709e15b\" (UID: \"5ec1cb1b-2a63-4f63-bf39-882ad709e15b\") " Aug 04 09:37:24.676835 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:24.676815 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ec1cb1b-2a63-4f63-bf39-882ad709e15b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ec1cb1b-2a63-4f63-bf39-882ad709e15b" (UID: "5ec1cb1b-2a63-4f63-bf39-882ad709e15b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:37:24.777554 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:24.777525 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec1cb1b-2a63-4f63-bf39-882ad709e15b-maas-api-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:25.552193 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:25.552152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" event={"ID":"5ec1cb1b-2a63-4f63-bf39-882ad709e15b","Type":"ContainerDied","Data":"1cd5d92ae455da03bc8415e496ec3fab80678a04872608ce638ca2db9c50c0f0"} Aug 04 09:37:25.552193 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:25.552180 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9" Aug 04 09:37:25.552446 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:25.552230 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1cd5d92ae455da03bc8415e496ec3fab80678a04872608ce638ca2db9c50c0f0" Aug 04 09:37:32.535351 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.535314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:32.567697 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.567667 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h"] Aug 04 09:37:32.567956 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.567930 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" podUID="438c5991-0220-4fa1-bdd8-1d2464573729" containerName="payload-processing" containerID="cri-o://5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0" gracePeriod=30 Aug 04 09:37:32.636555 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.636521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:32.722803 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.722770 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9"] Aug 04 09:37:32.726176 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.726137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:32.726675 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.726656 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e842246f-5f7f9"] Aug 04 09:37:32.789643 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.789563 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:32.823334 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.823310 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:37:32.837229 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.837198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438c5991-0220-4fa1-bdd8-1d2464573729-plugins-config-volume\") pod \"438c5991-0220-4fa1-bdd8-1d2464573729\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " Aug 04 09:37:32.837414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.837273 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksltk\" (UniqueName: \"kubernetes.io/projected/438c5991-0220-4fa1-bdd8-1d2464573729-kube-api-access-ksltk\") pod \"438c5991-0220-4fa1-bdd8-1d2464573729\" (UID: \"438c5991-0220-4fa1-bdd8-1d2464573729\") " Aug 04 09:37:32.837816 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.837779 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/438c5991-0220-4fa1-bdd8-1d2464573729-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "438c5991-0220-4fa1-bdd8-1d2464573729" (UID: "438c5991-0220-4fa1-bdd8-1d2464573729"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:37:32.840396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.840372 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/438c5991-0220-4fa1-bdd8-1d2464573729-kube-api-access-ksltk" (OuterVolumeSpecName: "kube-api-access-ksltk") pod "438c5991-0220-4fa1-bdd8-1d2464573729" (UID: "438c5991-0220-4fa1-bdd8-1d2464573729"). InnerVolumeSpecName "kube-api-access-ksltk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:37:32.884496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.883673 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg"] Aug 04 09:37:32.884496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.883937 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" containerID="cri-o://56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6" gracePeriod=30 Aug 04 09:37:32.906136 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.906108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:32.938658 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.938633 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438c5991-0220-4fa1-bdd8-1d2464573729-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:32.938658 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:32.938656 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksltk\" (UniqueName: \"kubernetes.io/projected/438c5991-0220-4fa1-bdd8-1d2464573729-kube-api-access-ksltk\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:33.206367 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.206336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:33.419119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.419082 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ec1cb1b-2a63-4f63-bf39-882ad709e15b" path="/var/lib/kubelet/pods/5ec1cb1b-2a63-4f63-bf39-882ad709e15b/volumes" Aug 04 09:37:33.519033 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.518930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:33.600407 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.600377 2562 generic.go:356] "Generic (PLEG): container finished" podID="438c5991-0220-4fa1-bdd8-1d2464573729" containerID="5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0" exitCode=0 Aug 04 09:37:33.600790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.600423 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" Aug 04 09:37:33.600790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.600461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" event={"ID":"438c5991-0220-4fa1-bdd8-1d2464573729","Type":"ContainerDied","Data":"5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0"} Aug 04 09:37:33.600790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.600517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h" event={"ID":"438c5991-0220-4fa1-bdd8-1d2464573729","Type":"ContainerDied","Data":"ba12dab68f86841898ba13d4e83f4cde4c7454795e8e6ed8c5738d35336844e4"} Aug 04 09:37:33.600790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.600547 2562 scope.go:117] "RemoveContainer" containerID="5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0" Aug 04 09:37:33.609430 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.609411 2562 scope.go:117] "RemoveContainer" containerID="5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0" Aug 04 09:37:33.609661 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:37:33.609646 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0\": container with ID starting with 5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0 not found: ID does not exist" containerID="5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0" Aug 04 09:37:33.609734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.609668 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0"} err="failed to get container status \"5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0\": rpc error: code = NotFound desc = could not find container \"5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0\": container with ID starting with 5a5638578baa747db32b695f24add54cd2605ffbd5de15360646916afb5bacb0 not found: ID does not exist" Aug 04 09:37:33.613434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.613405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:33.624235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.624210 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h"] Aug 04 09:37:33.631291 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.631273 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e842246f-78ffcbbc6b-8rc7h"] Aug 04 09:37:33.769884 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:33.769773 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:34.118464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118383 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss"] Aug 04 09:37:34.118781 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118769 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec1cb1b-2a63-4f63-bf39-882ad709e15b" containerName="revoke-keys" Aug 04 09:37:34.118823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118783 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec1cb1b-2a63-4f63-bf39-882ad709e15b" containerName="revoke-keys" Aug 04 09:37:34.118823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118802 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="438c5991-0220-4fa1-bdd8-1d2464573729" containerName="payload-processing" Aug 04 09:37:34.118823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118807 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="438c5991-0220-4fa1-bdd8-1d2464573729" containerName="payload-processing" Aug 04 09:37:34.118920 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118869 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec1cb1b-2a63-4f63-bf39-882ad709e15b" containerName="revoke-keys" Aug 04 09:37:34.118920 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.118881 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="438c5991-0220-4fa1-bdd8-1d2464573729" containerName="payload-processing" Aug 04 09:37:34.123226 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.123208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.125960 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.125940 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-93164817-gw-openshift-default-dockercfg-8gghx\"" Aug 04 09:37:34.145660 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.145638 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss"] Aug 04 09:37:34.149430 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-envoy\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149511 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d971ae6a-7712-4451-b364-254bba4d128f-istio-podinfo\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149511 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-certs\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-credential-socket\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-data\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d971ae6a-7712-4451-b364-254bba4d128f-istiod-ca-cert\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149679 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-socket\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149724 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8znnk\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-kube-api-access-8znnk\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.149764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.149729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-istio-token\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.166627 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.166601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:34.250322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-certs\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-credential-socket\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-data\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d971ae6a-7712-4451-b364-254bba4d128f-istiod-ca-cert\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-socket\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znnk\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-kube-api-access-8znnk\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-istio-token\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250513 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-envoy\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d971ae6a-7712-4451-b364-254bba4d128f-istio-podinfo\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250717 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-certs\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250853 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-credential-socket\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-data\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.250971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.250941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-socket\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.251171 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.251154 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d971ae6a-7712-4451-b364-254bba4d128f-istiod-ca-cert\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.252804 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.252781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-envoy\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.252942 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.252923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d971ae6a-7712-4451-b364-254bba4d128f-istio-podinfo\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.258138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.258117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-istio-token\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.258896 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.258868 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8znnk\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-kube-api-access-8znnk\") pod \"e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.356027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.355994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:34.433034 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.432998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:34.572581 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.572551 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss"] Aug 04 09:37:34.574228 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:37:34.574203 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd971ae6a_7712_4451_b364_254bba4d128f.slice/crio-d5b4be7be17686c3c7780a4b97fc2e297c87eda8f97b103151db75b1f6c7473f WatchSource:0}: Error finding container d5b4be7be17686c3c7780a4b97fc2e297c87eda8f97b103151db75b1f6c7473f: Status 404 returned error can't find the container with id d5b4be7be17686c3c7780a4b97fc2e297c87eda8f97b103151db75b1f6c7473f Aug 04 09:37:34.576383 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.576355 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:34.576464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.576421 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:34.576464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.576450 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:34.607931 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:34.607900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" event={"ID":"d971ae6a-7712-4451-b364-254bba4d128f","Type":"ContainerStarted","Data":"d5b4be7be17686c3c7780a4b97fc2e297c87eda8f97b103151db75b1f6c7473f"} Aug 04 09:37:35.416452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:35.416408 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="438c5991-0220-4fa1-bdd8-1d2464573729" path="/var/lib/kubelet/pods/438c5991-0220-4fa1-bdd8-1d2464573729/volumes" Aug 04 09:37:35.616324 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:35.616282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" event={"ID":"d971ae6a-7712-4451-b364-254bba4d128f","Type":"ContainerStarted","Data":"46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9"} Aug 04 09:37:35.639369 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:35.639314 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podStartSLOduration=1.63929908 podStartE2EDuration="1.63929908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:37:35.636067806 +0000 UTC m=+1772.768277374" watchObservedRunningTime="2026-08-04 09:37:35.63929908 +0000 UTC m=+1772.771508611" Aug 04 09:37:36.434071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:36.434032 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:36.439158 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:36.439133 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:36.622676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:36.622647 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:36.623680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:36.623659 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:37.171460 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:37.171423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:37.417078 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:37.417048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:37.517274 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:37.517184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:37.913322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:37.913296 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:38.013283 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.013225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:38.134965 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.134938 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:38.189092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189023 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-certs\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189083 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-envoy\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189113 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-istio-token\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded2346f-5281-42c4-8e92-647043657be2-istio-podinfo\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189174 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded2346f-5281-42c4-8e92-647043657be2-istiod-ca-cert\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189189 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-socket\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189235 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-credential-socket\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189295 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zfrx\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-kube-api-access-8zfrx\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189341 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-data\") pod \"ded2346f-5281-42c4-8e92-647043657be2\" (UID: \"ded2346f-5281-42c4-8e92-647043657be2\") " Aug 04 09:37:38.189638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189353 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:37:38.189638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189590 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.189811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189669 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:37:38.189811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189677 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:37:38.189811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189691 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:37:38.189811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.189634 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded2346f-5281-42c4-8e92-647043657be2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:37:38.191471 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.191443 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-kube-api-access-8zfrx" (OuterVolumeSpecName: "kube-api-access-8zfrx") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "kube-api-access-8zfrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:37:38.191471 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.191467 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:37:38.191734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.191715 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ded2346f-5281-42c4-8e92-647043657be2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:37:38.191789 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.191749 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ded2346f-5281-42c4-8e92-647043657be2" (UID: "ded2346f-5281-42c4-8e92-647043657be2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:37:38.290654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290620 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zfrx\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-kube-api-access-8zfrx\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290647 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290662 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290674 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded2346f-5281-42c4-8e92-647043657be2-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290686 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded2346f-5281-42c4-8e92-647043657be2-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290697 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded2346f-5281-42c4-8e92-647043657be2-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290706 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.290879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.290716 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded2346f-5281-42c4-8e92-647043657be2-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:37:38.635546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.635460 2562 generic.go:356] "Generic (PLEG): container finished" podID="ded2346f-5281-42c4-8e92-647043657be2" containerID="56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6" exitCode=0 Aug 04 09:37:38.635546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.635519 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" Aug 04 09:37:38.635724 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.635540 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" event={"ID":"ded2346f-5281-42c4-8e92-647043657be2","Type":"ContainerDied","Data":"56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6"} Aug 04 09:37:38.635724 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.635593 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg" event={"ID":"ded2346f-5281-42c4-8e92-647043657be2","Type":"ContainerDied","Data":"be0090582767238f368b76c7c76f6028ad09d88f267c0a157fa4c358424e8c1b"} Aug 04 09:37:38.635724 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.635615 2562 scope.go:117] "RemoveContainer" containerID="56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6" Aug 04 09:37:38.646119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.646102 2562 scope.go:117] "RemoveContainer" containerID="675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084" Aug 04 09:37:38.654107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.654084 2562 scope.go:117] "RemoveContainer" containerID="56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6" Aug 04 09:37:38.654375 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:37:38.654354 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6\": container with ID starting with 56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6 not found: ID does not exist" containerID="56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6" Aug 04 09:37:38.654452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.654386 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6"} err="failed to get container status \"56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6\": rpc error: code = NotFound desc = could not find container \"56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6\": container with ID starting with 56332543c7b415d729af4f67717f90836202d032d7daf09ee9bfa254e6bd22a6 not found: ID does not exist" Aug 04 09:37:38.654452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.654402 2562 scope.go:117] "RemoveContainer" containerID="675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084" Aug 04 09:37:38.654636 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:37:38.654613 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084\": container with ID starting with 675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084 not found: ID does not exist" containerID="675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084" Aug 04 09:37:38.654693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.654642 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084"} err="failed to get container status \"675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084\": rpc error: code = NotFound desc = could not find container \"675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084\": container with ID starting with 675b13fc34958d692a28e71b0823d9d5fde474e04e203546444e34be9793c084 not found: ID does not exist" Aug 04 09:37:38.657672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.657653 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg"] Aug 04 09:37:38.661663 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:38.661642 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e842246f-openshift-default-85dd76dcfd-k79vg"] Aug 04 09:37:39.416208 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:39.416173 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ded2346f-5281-42c4-8e92-647043657be2" path="/var/lib/kubelet/pods/ded2346f-5281-42c4-8e92-647043657be2/volumes" Aug 04 09:37:49.707242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:49.707207 2562 generic.go:356] "Generic (PLEG): container finished" podID="d971ae6a-7712-4451-b364-254bba4d128f" containerID="46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9" exitCode=0 Aug 04 09:37:49.707677 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:49.707285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" event={"ID":"d971ae6a-7712-4451-b364-254bba4d128f","Type":"ContainerDied","Data":"46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9"} Aug 04 09:37:49.707725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:49.707699 2562 scope.go:117] "RemoveContainer" containerID="46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9" Aug 04 09:37:49.708705 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:49.708681 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:49.708778 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:49.708750 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:49.708814 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:49.708781 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:37:50.713236 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:50.713203 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:37:50.715486 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:50.715460 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" event={"ID":"d971ae6a-7712-4451-b364-254bba4d128f","Type":"ContainerStarted","Data":"e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad"} Aug 04 09:37:51.434099 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:51.434067 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:51.435299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:51.435275 2562 patch_prober.go:28] interesting pod/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 09:37:51.435423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:51.435319 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 09:37:51.622906 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:51.622872 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:52.433880 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:52.433841 2562 patch_prober.go:28] interesting pod/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 09:37:52.434368 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:52.433899 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 09:37:53.433519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:53.433480 2562 patch_prober.go:28] interesting pod/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 09:37:53.433719 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:53.433551 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 09:37:54.433599 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:54.433558 2562 patch_prober.go:28] interesting pod/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 09:37:54.434002 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:54.433615 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 09:37:55.433703 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:55.433619 2562 patch_prober.go:28] interesting pod/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 04 09:37:55.433703 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:55.433670 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 04 09:37:56.437993 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:56.437960 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:37:56.438776 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:37:56.438753 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:38:17.976179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:17.976143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:18.050157 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.050123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:18.159392 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.159351 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:18.311970 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.311892 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss"] Aug 04 09:38:18.312182 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.312156 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" containerID="cri-o://e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad" gracePeriod=30 Aug 04 09:38:18.328952 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.328925 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:18.597923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.597840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:18.883220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:18.883139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:19.242088 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:19.242048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:19.786923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:19.786889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:21.707708 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:21.707674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:21.803288 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:21.803246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:22.839303 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:22.839242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:22.999577 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:22.999545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:23.478070 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.478043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:23.557668 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.557645 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:38:23.583028 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.582925 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:23.613874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.613846 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-istio-token\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.613893 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-envoy\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.613946 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-credential-socket\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.613982 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d971ae6a-7712-4451-b364-254bba4d128f-istio-podinfo\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614002 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-data\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614031 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d971ae6a-7712-4451-b364-254bba4d128f-istiod-ca-cert\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614067 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-certs\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614366 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614228 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:38:23.614429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614361 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:38:23.614429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614373 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:38:23.614429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614404 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-socket\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614593 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614442 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d971ae6a-7712-4451-b364-254bba4d128f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:38:23.614593 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614457 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8znnk\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-kube-api-access-8znnk\") pod \"d971ae6a-7712-4451-b364-254bba4d128f\" (UID: \"d971ae6a-7712-4451-b364-254bba4d128f\") " Aug 04 09:38:23.614711 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614615 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:38:23.614775 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614757 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.614775 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614771 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.614874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614780 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d971ae6a-7712-4451-b364-254bba4d128f-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.614874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614788 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.614874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.614796 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.616015 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.615995 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:38:23.616314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.616297 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d971ae6a-7712-4451-b364-254bba4d128f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:38:23.616488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.616468 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:38:23.616694 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.616674 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-kube-api-access-8znnk" (OuterVolumeSpecName: "kube-api-access-8znnk") pod "d971ae6a-7712-4451-b364-254bba4d128f" (UID: "d971ae6a-7712-4451-b364-254bba4d128f"). InnerVolumeSpecName "kube-api-access-8znnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:38:23.652115 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652089 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv"] Aug 04 09:38:23.652521 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652506 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" Aug 04 09:38:23.652564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652522 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" Aug 04 09:38:23.652564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652545 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" Aug 04 09:38:23.652564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652551 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" Aug 04 09:38:23.652564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652560 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" Aug 04 09:38:23.652564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652566 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" Aug 04 09:38:23.652728 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652619 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" Aug 04 09:38:23.652728 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652628 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded2346f-5281-42c4-8e92-647043657be2" containerName="istio-proxy" Aug 04 09:38:23.652728 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.652634 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" Aug 04 09:38:23.655938 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.655920 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.658223 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.658205 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d6e1c0f5\"" Aug 04 09:38:23.658350 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.658318 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d6e1c0f5-dockercfg-92pdx\"" Aug 04 09:38:23.666646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.666623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv"] Aug 04 09:38:23.716121 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.716090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67wz7\" (UniqueName: \"kubernetes.io/projected/db5ef902-3340-4d8f-8800-42fc69b105f8-kube-api-access-67wz7\") pod \"payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.716305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.716138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db5ef902-3340-4d8f-8800-42fc69b105f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.716305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.716240 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d971ae6a-7712-4451-b364-254bba4d128f-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.716305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.716266 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d971ae6a-7712-4451-b364-254bba4d128f-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.716305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.716275 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8znnk\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-kube-api-access-8znnk\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.716305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.716283 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d971ae6a-7712-4451-b364-254bba4d128f-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:23.729961 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.729933 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t"] Aug 04 09:38:23.730335 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.730321 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" Aug 04 09:38:23.730390 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.730336 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" Aug 04 09:38:23.730427 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.730408 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d971ae6a-7712-4451-b364-254bba4d128f" containerName="istio-proxy" Aug 04 09:38:23.733462 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.733447 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.744621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.744599 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t"] Aug 04 09:38:23.817353 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.817323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67wz7\" (UniqueName: \"kubernetes.io/projected/db5ef902-3340-4d8f-8800-42fc69b105f8-kube-api-access-67wz7\") pod \"payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.817489 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.817374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-plugins-config-volume\") pod \"payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.817548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.817481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlpv7\" (UniqueName: \"kubernetes.io/projected/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-kube-api-access-qlpv7\") pod \"payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.817548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.817522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db5ef902-3340-4d8f-8800-42fc69b105f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.818156 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.818132 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db5ef902-3340-4d8f-8800-42fc69b105f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.826414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.826395 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67wz7\" (UniqueName: \"kubernetes.io/projected/db5ef902-3340-4d8f-8800-42fc69b105f8-kube-api-access-67wz7\") pod \"payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:23.904725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.904697 2562 generic.go:356] "Generic (PLEG): container finished" podID="d971ae6a-7712-4451-b364-254bba4d128f" containerID="e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad" exitCode=0 Aug 04 09:38:23.905165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.904763 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" Aug 04 09:38:23.905165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.904782 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" event={"ID":"d971ae6a-7712-4451-b364-254bba4d128f","Type":"ContainerDied","Data":"e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad"} Aug 04 09:38:23.905165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.904849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss" event={"ID":"d971ae6a-7712-4451-b364-254bba4d128f","Type":"ContainerDied","Data":"d5b4be7be17686c3c7780a4b97fc2e297c87eda8f97b103151db75b1f6c7473f"} Aug 04 09:38:23.905165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.904872 2562 scope.go:117] "RemoveContainer" containerID="e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad" Aug 04 09:38:23.914198 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.914182 2562 scope.go:117] "RemoveContainer" containerID="46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9" Aug 04 09:38:23.918754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.918724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-plugins-config-volume\") pod \"payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.918858 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.918761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlpv7\" (UniqueName: \"kubernetes.io/projected/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-kube-api-access-qlpv7\") pod \"payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.919429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.919409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-plugins-config-volume\") pod \"payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.922329 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.922313 2562 scope.go:117] "RemoveContainer" containerID="e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad" Aug 04 09:38:23.922554 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:38:23.922535 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad\": container with ID starting with e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad not found: ID does not exist" containerID="e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad" Aug 04 09:38:23.922629 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.922568 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad"} err="failed to get container status \"e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad\": rpc error: code = NotFound desc = could not find container \"e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad\": container with ID starting with e4a45f1824b2fd82cabb7565830b714836736c2ced0f394b9ce9c8c6e50a29ad not found: ID does not exist" Aug 04 09:38:23.922629 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.922593 2562 scope.go:117] "RemoveContainer" containerID="46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9" Aug 04 09:38:23.922832 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:38:23.922814 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9\": container with ID starting with 46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9 not found: ID does not exist" containerID="46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9" Aug 04 09:38:23.922869 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.922838 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9"} err="failed to get container status \"46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9\": rpc error: code = NotFound desc = could not find container \"46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9\": container with ID starting with 46a43daa6aeb6caccf8cbd904d858e36319bbbfd46b6a8a4a7e1db743e0c3da9 not found: ID does not exist" Aug 04 09:38:23.927560 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.927539 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss"] Aug 04 09:38:23.928760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.928736 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlpv7\" (UniqueName: \"kubernetes.io/projected/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-kube-api-access-qlpv7\") pod \"payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:23.931546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.931529 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-93164817-gw-openshift-default-5bfb464cdd-s2pss"] Aug 04 09:38:23.971540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:23.971512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:24.043850 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.043822 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:24.110132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.110104 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv"] Aug 04 09:38:24.112347 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:38:24.112320 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5ef902_3340_4d8f_8800_42fc69b105f8.slice/crio-e56c96a76930a3164fad8dbe8fd579b57498b608b81a04a5338cbcdc5ab60fcd WatchSource:0}: Error finding container e56c96a76930a3164fad8dbe8fd579b57498b608b81a04a5338cbcdc5ab60fcd: Status 404 returned error can't find the container with id e56c96a76930a3164fad8dbe8fd579b57498b608b81a04a5338cbcdc5ab60fcd Aug 04 09:38:24.181746 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.181709 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t"] Aug 04 09:38:24.183516 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:38:24.183490 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcf8bd8_a7a0_4a28_aec3_f83efa119d7a.slice/crio-f8fa1bb097bc8df970a83c598cfdc47e6fd6965a9ea9d0e21f5d01bf1780f317 WatchSource:0}: Error finding container f8fa1bb097bc8df970a83c598cfdc47e6fd6965a9ea9d0e21f5d01bf1780f317: Status 404 returned error can't find the container with id f8fa1bb097bc8df970a83c598cfdc47e6fd6965a9ea9d0e21f5d01bf1780f317 Aug 04 09:38:24.911815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.911777 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" event={"ID":"db5ef902-3340-4d8f-8800-42fc69b105f8","Type":"ContainerStarted","Data":"02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e"} Aug 04 09:38:24.912318 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.911836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" event={"ID":"db5ef902-3340-4d8f-8800-42fc69b105f8","Type":"ContainerStarted","Data":"e56c96a76930a3164fad8dbe8fd579b57498b608b81a04a5338cbcdc5ab60fcd"} Aug 04 09:38:24.912318 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.911893 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:24.914203 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.914182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" event={"ID":"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a","Type":"ContainerStarted","Data":"d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90"} Aug 04 09:38:24.914317 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.914215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" event={"ID":"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a","Type":"ContainerStarted","Data":"f8fa1bb097bc8df970a83c598cfdc47e6fd6965a9ea9d0e21f5d01bf1780f317"} Aug 04 09:38:24.914414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.914319 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:24.928580 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.928538 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" podStartSLOduration=1.928524802 podStartE2EDuration="1.928524802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:38:24.927503389 +0000 UTC m=+1822.059712921" watchObservedRunningTime="2026-08-04 09:38:24.928524802 +0000 UTC m=+1822.060734315" Aug 04 09:38:24.946219 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:24.946160 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" podStartSLOduration=1.9461445419999999 podStartE2EDuration="1.946144542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:38:24.943633968 +0000 UTC m=+1822.075843503" watchObservedRunningTime="2026-08-04 09:38:24.946144542 +0000 UTC m=+1822.078354075" Aug 04 09:38:25.416897 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:25.416860 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d971ae6a-7712-4451-b364-254bba4d128f" path="/var/lib/kubelet/pods/d971ae6a-7712-4451-b364-254bba4d128f/volumes" Aug 04 09:38:27.889441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:27.889408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:35.921728 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:35.921690 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:35.922302 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:35.921767 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:47.926133 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:47.926093 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht"] Aug 04 09:38:47.929575 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:47.929556 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:47.932415 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:47.932396 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:38:47.932763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:47.932745 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 09:38:47.942140 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:47.942117 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht"] Aug 04 09:38:48.036779 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:48.036740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd077b1-71de-4e75-bd74-da0e30a5efcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht\" (UID: \"fcd077b1-71de-4e75-bd74-da0e30a5efcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:48.137201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:48.137169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd077b1-71de-4e75-bd74-da0e30a5efcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht\" (UID: \"fcd077b1-71de-4e75-bd74-da0e30a5efcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:48.137937 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:48.137914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd077b1-71de-4e75-bd74-da0e30a5efcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht\" (UID: \"fcd077b1-71de-4e75-bd74-da0e30a5efcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:48.242516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:48.242443 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:48.580357 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:48.580329 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht"] Aug 04 09:38:48.581822 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:38:48.581797 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd077b1_71de_4e75_bd74_da0e30a5efcc.slice/crio-8953cd13c2b6d00ae8e687c07a1f4c524de5005f29206b744f669b063250cc59 WatchSource:0}: Error finding container 8953cd13c2b6d00ae8e687c07a1f4c524de5005f29206b744f669b063250cc59: Status 404 returned error can't find the container with id 8953cd13c2b6d00ae8e687c07a1f4c524de5005f29206b744f669b063250cc59 Aug 04 09:38:49.051594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:49.051549 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" event={"ID":"fcd077b1-71de-4e75-bd74-da0e30a5efcc","Type":"ContainerStarted","Data":"23810956e600d2c33322ae361fd3d8c66b3af86bf6a2df0534d5a65dd2c476d2"} Aug 04 09:38:49.052007 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:49.051608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" event={"ID":"fcd077b1-71de-4e75-bd74-da0e30a5efcc","Type":"ContainerStarted","Data":"8953cd13c2b6d00ae8e687c07a1f4c524de5005f29206b744f669b063250cc59"} Aug 04 09:38:49.073740 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:49.073690 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" podStartSLOduration=2.073674055 podStartE2EDuration="2.073674055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:38:49.073435452 +0000 UTC m=+1846.205644986" watchObservedRunningTime="2026-08-04 09:38:49.073674055 +0000 UTC m=+1846.205883586" Aug 04 09:38:50.058076 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:50.058040 2562 generic.go:356] "Generic (PLEG): container finished" podID="fcd077b1-71de-4e75-bd74-da0e30a5efcc" containerID="23810956e600d2c33322ae361fd3d8c66b3af86bf6a2df0534d5a65dd2c476d2" exitCode=0 Aug 04 09:38:50.058622 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:50.058084 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" event={"ID":"fcd077b1-71de-4e75-bd74-da0e30a5efcc","Type":"ContainerDied","Data":"23810956e600d2c33322ae361fd3d8c66b3af86bf6a2df0534d5a65dd2c476d2"} Aug 04 09:38:51.193081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:51.193053 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:51.366028 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:51.365947 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd077b1-71de-4e75-bd74-da0e30a5efcc-maas-api-service-ca\") pod \"fcd077b1-71de-4e75-bd74-da0e30a5efcc\" (UID: \"fcd077b1-71de-4e75-bd74-da0e30a5efcc\") " Aug 04 09:38:51.366343 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:51.366320 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd077b1-71de-4e75-bd74-da0e30a5efcc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcd077b1-71de-4e75-bd74-da0e30a5efcc" (UID: "fcd077b1-71de-4e75-bd74-da0e30a5efcc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:38:51.467507 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:51.467477 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd077b1-71de-4e75-bd74-da0e30a5efcc-maas-api-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:52.070476 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:52.070436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" event={"ID":"fcd077b1-71de-4e75-bd74-da0e30a5efcc","Type":"ContainerDied","Data":"8953cd13c2b6d00ae8e687c07a1f4c524de5005f29206b744f669b063250cc59"} Aug 04 09:38:52.070476 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:52.070453 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht" Aug 04 09:38:52.070691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:52.070485 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8953cd13c2b6d00ae8e687c07a1f4c524de5005f29206b744f669b063250cc59" Aug 04 09:38:53.467619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.467587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:53.475289 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.475247 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t"] Aug 04 09:38:53.475475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.475453 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" podUID="1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" containerName="payload-processing" containerID="cri-o://d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90" gracePeriod=30 Aug 04 09:38:53.483498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.483472 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv"] Aug 04 09:38:53.483837 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.483805 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" podUID="db5ef902-3340-4d8f-8800-42fc69b105f8" containerName="payload-pre-processing" containerID="cri-o://02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e" gracePeriod=30 Aug 04 09:38:53.563199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.563168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:53.660003 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.659966 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht"] Aug 04 09:38:53.664726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.664700 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d6e1c0f5-b7pht"] Aug 04 09:38:53.682372 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.682311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:53.726881 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.726825 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:53.750558 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.750535 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:53.888421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.888391 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-plugins-config-volume\") pod \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " Aug 04 09:38:53.888421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.888424 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db5ef902-3340-4d8f-8800-42fc69b105f8-plugins-config-volume\") pod \"db5ef902-3340-4d8f-8800-42fc69b105f8\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " Aug 04 09:38:53.888636 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.888459 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlpv7\" (UniqueName: \"kubernetes.io/projected/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-kube-api-access-qlpv7\") pod \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\" (UID: \"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a\") " Aug 04 09:38:53.888636 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.888510 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67wz7\" (UniqueName: \"kubernetes.io/projected/db5ef902-3340-4d8f-8800-42fc69b105f8-kube-api-access-67wz7\") pod \"db5ef902-3340-4d8f-8800-42fc69b105f8\" (UID: \"db5ef902-3340-4d8f-8800-42fc69b105f8\") " Aug 04 09:38:53.888774 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.888753 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" (UID: "1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:38:53.888838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.888815 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db5ef902-3340-4d8f-8800-42fc69b105f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db5ef902-3340-4d8f-8800-42fc69b105f8" (UID: "db5ef902-3340-4d8f-8800-42fc69b105f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:38:53.890590 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.890570 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-kube-api-access-qlpv7" (OuterVolumeSpecName: "kube-api-access-qlpv7") pod "1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" (UID: "1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a"). InnerVolumeSpecName "kube-api-access-qlpv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:38:53.890669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.890603 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5ef902-3340-4d8f-8800-42fc69b105f8-kube-api-access-67wz7" (OuterVolumeSpecName: "kube-api-access-67wz7") pod "db5ef902-3340-4d8f-8800-42fc69b105f8" (UID: "db5ef902-3340-4d8f-8800-42fc69b105f8"). InnerVolumeSpecName "kube-api-access-67wz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:38:53.989220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.989148 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67wz7\" (UniqueName: \"kubernetes.io/projected/db5ef902-3340-4d8f-8800-42fc69b105f8-kube-api-access-67wz7\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:53.989220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.989174 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:53.989220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.989185 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db5ef902-3340-4d8f-8800-42fc69b105f8-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:53.989220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:53.989194 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlpv7\" (UniqueName: \"kubernetes.io/projected/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a-kube-api-access-qlpv7\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:38:54.081987 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.081949 2562 generic.go:356] "Generic (PLEG): container finished" podID="db5ef902-3340-4d8f-8800-42fc69b105f8" containerID="02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e" exitCode=0 Aug 04 09:38:54.082152 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.081995 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" Aug 04 09:38:54.082152 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.082001 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" event={"ID":"db5ef902-3340-4d8f-8800-42fc69b105f8","Type":"ContainerDied","Data":"02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e"} Aug 04 09:38:54.082152 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.082063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv" event={"ID":"db5ef902-3340-4d8f-8800-42fc69b105f8","Type":"ContainerDied","Data":"e56c96a76930a3164fad8dbe8fd579b57498b608b81a04a5338cbcdc5ab60fcd"} Aug 04 09:38:54.082152 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.082090 2562 scope.go:117] "RemoveContainer" containerID="02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e" Aug 04 09:38:54.083538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.083518 2562 generic.go:356] "Generic (PLEG): container finished" podID="1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" containerID="d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90" exitCode=0 Aug 04 09:38:54.083646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.083563 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" Aug 04 09:38:54.083646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.083595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" event={"ID":"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a","Type":"ContainerDied","Data":"d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90"} Aug 04 09:38:54.083743 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.083647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t" event={"ID":"1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a","Type":"ContainerDied","Data":"f8fa1bb097bc8df970a83c598cfdc47e6fd6965a9ea9d0e21f5d01bf1780f317"} Aug 04 09:38:54.092975 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.092954 2562 scope.go:117] "RemoveContainer" containerID="02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e" Aug 04 09:38:54.093377 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:38:54.093202 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e\": container with ID starting with 02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e not found: ID does not exist" containerID="02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e" Aug 04 09:38:54.093377 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.093228 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e"} err="failed to get container status \"02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e\": rpc error: code = NotFound desc = could not find container \"02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e\": container with ID starting with 02af13eca6d0e407f2982be1c3a4ea588f37dd3cb14df9c7c885c34245fb776e not found: ID does not exist" Aug 04 09:38:54.093377 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.093244 2562 scope.go:117] "RemoveContainer" containerID="d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90" Aug 04 09:38:54.101568 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.101550 2562 scope.go:117] "RemoveContainer" containerID="d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90" Aug 04 09:38:54.101803 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:38:54.101786 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90\": container with ID starting with d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90 not found: ID does not exist" containerID="d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90" Aug 04 09:38:54.101848 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.101809 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90"} err="failed to get container status \"d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90\": rpc error: code = NotFound desc = could not find container \"d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90\": container with ID starting with d98a3f5f66363444c9c96745a95298c17f9ec8dc3f792c707aa0357c82b92a90 not found: ID does not exist" Aug 04 09:38:54.105140 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.105120 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv"] Aug 04 09:38:54.111202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.111180 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d6e1c0f5-5cff9df676-628sv"] Aug 04 09:38:54.124899 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.124876 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t"] Aug 04 09:38:54.129383 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:54.129363 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d6e1c0f5-648f55dcd4-qqj8t"] Aug 04 09:38:55.044065 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.044031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:55.298112 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.298030 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:55.415939 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.415902 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" path="/var/lib/kubelet/pods/1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a/volumes" Aug 04 09:38:55.416273 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.416240 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db5ef902-3340-4d8f-8800-42fc69b105f8" path="/var/lib/kubelet/pods/db5ef902-3340-4d8f-8800-42fc69b105f8/volumes" Aug 04 09:38:55.416573 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.416560 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcd077b1-71de-4e75-bd74-da0e30a5efcc" path="/var/lib/kubelet/pods/fcd077b1-71de-4e75-bd74-da0e30a5efcc/volumes" Aug 04 09:38:55.548462 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.548378 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:55.859450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859375 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js"] Aug 04 09:38:55.859830 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859793 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5ef902-3340-4d8f-8800-42fc69b105f8" containerName="payload-pre-processing" Aug 04 09:38:55.859894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859831 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5ef902-3340-4d8f-8800-42fc69b105f8" containerName="payload-pre-processing" Aug 04 09:38:55.859894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859841 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd077b1-71de-4e75-bd74-da0e30a5efcc" containerName="revoke-keys" Aug 04 09:38:55.859894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859847 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd077b1-71de-4e75-bd74-da0e30a5efcc" containerName="revoke-keys" Aug 04 09:38:55.859894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859861 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" containerName="payload-processing" Aug 04 09:38:55.859894 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859867 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" containerName="payload-processing" Aug 04 09:38:55.860053 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859939 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf8bd8-a7a0-4a28-aec3-f83efa119d7a" containerName="payload-processing" Aug 04 09:38:55.860053 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859947 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd077b1-71de-4e75-bd74-da0e30a5efcc" containerName="revoke-keys" Aug 04 09:38:55.860053 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.859954 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="db5ef902-3340-4d8f-8800-42fc69b105f8" containerName="payload-pre-processing" Aug 04 09:38:55.864374 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.864352 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:55.867472 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.867454 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-cf8d7038-openshift-default-dockercfg-k8ghl\"" Aug 04 09:38:55.879925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.879537 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js"] Aug 04 09:38:55.885917 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:55.885894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:56.007498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-envoy\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007498 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g85hn\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-kube-api-access-g85hn\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d579179-f2c3-40b1-b204-3db8c970d78a-istiod-ca-cert\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-podinfo\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-certs\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-token\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007691 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-socket\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007714 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-data\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.007886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.007734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-credential-socket\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.108759 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.108721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-envoy\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.108759 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.108760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g85hn\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-kube-api-access-g85hn\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.108880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d579179-f2c3-40b1-b204-3db8c970d78a-istiod-ca-cert\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.108919 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-podinfo\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.108988 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-certs\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-token\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109075 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-socket\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-data\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-credential-socket\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109407 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-certs\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109499 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-socket\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109554 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-credential-socket\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109726 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d579179-f2c3-40b1-b204-3db8c970d78a-istiod-ca-cert\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.109885 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.109730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-data\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.111098 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.111076 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-envoy\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.111323 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.111307 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-podinfo\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.116533 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.116512 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-token\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.116622 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.116561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g85hn\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-kube-api-access-g85hn\") pod \"e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.174125 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.174093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:56.174560 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.174514 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:56.312123 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.312097 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js"] Aug 04 09:38:56.314609 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:38:56.314580 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d579179_f2c3_40b1_b204_3db8c970d78a.slice/crio-7f00484620745b5316b929b4d472464ffd9996a9627870e35a7e6acf4d8ffa58 WatchSource:0}: Error finding container 7f00484620745b5316b929b4d472464ffd9996a9627870e35a7e6acf4d8ffa58: Status 404 returned error can't find the container with id 7f00484620745b5316b929b4d472464ffd9996a9627870e35a7e6acf4d8ffa58 Aug 04 09:38:56.316666 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.316636 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:38:56.316762 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.316694 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:38:56.316762 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:56.316723 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:38:57.103135 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:57.103100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" event={"ID":"3d579179-f2c3-40b1-b204-3db8c970d78a","Type":"ContainerStarted","Data":"4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87"} Aug 04 09:38:57.103330 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:57.103151 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" event={"ID":"3d579179-f2c3-40b1-b204-3db8c970d78a","Type":"ContainerStarted","Data":"7f00484620745b5316b929b4d472464ffd9996a9627870e35a7e6acf4d8ffa58"} Aug 04 09:38:57.126773 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:57.126722 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podStartSLOduration=2.126707818 podStartE2EDuration="2.126707818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:38:57.123802254 +0000 UTC m=+1854.256011786" watchObservedRunningTime="2026-08-04 09:38:57.126707818 +0000 UTC m=+1854.258917349" Aug 04 09:38:57.174431 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:57.174401 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:57.179563 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:57.179538 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:58.109542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:58.109513 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:58.110514 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:58.110491 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:38:58.552094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:58.552063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:58.651542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:58.651512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:59.718457 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:59.718421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:38:59.889548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:38:59.889516 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:00.371424 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:00.371390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:00.471864 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:00.471833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:05.123722 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:05.123686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:12.208031 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:12.207992 2562 generic.go:356] "Generic (PLEG): container finished" podID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerID="4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87" exitCode=0 Aug 04 09:39:12.208453 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:12.208066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" event={"ID":"3d579179-f2c3-40b1-b204-3db8c970d78a","Type":"ContainerDied","Data":"4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87"} Aug 04 09:39:12.208509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:12.208496 2562 scope.go:117] "RemoveContainer" containerID="4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87" Aug 04 09:39:12.209564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:12.209536 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:39:12.209672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:12.209626 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:39:12.209745 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:12.209673 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:39:13.109808 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:13.109774 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:13.215917 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:13.215878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" event={"ID":"3d579179-f2c3-40b1-b204-3db8c970d78a","Type":"ContainerStarted","Data":"a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8"} Aug 04 09:39:14.174677 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:14.174634 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:14.175789 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:14.175758 2562 patch_prober.go:28] interesting pod/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 04 09:39:14.175951 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:14.175812 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 04 09:39:15.174447 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:15.174413 2562 patch_prober.go:28] interesting pod/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 04 09:39:15.174924 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:15.174477 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 04 09:39:16.174827 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:16.174789 2562 patch_prober.go:28] interesting pod/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 04 09:39:16.175213 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:16.174843 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 04 09:39:17.174510 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:17.174473 2562 patch_prober.go:28] interesting pod/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 04 09:39:17.174701 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:17.174547 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 04 09:39:18.175017 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:18.174979 2562 patch_prober.go:28] interesting pod/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 04 09:39:18.175465 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:18.175047 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 04 09:39:19.178468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:19.178435 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:19.184898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:19.179293 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:19.185669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:19.185645 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:30.338559 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.338485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:30.524431 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.524399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:30.594658 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.594576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:30.669898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.669852 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js"] Aug 04 09:39:30.670188 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.670159 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" containerID="cri-o://a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8" gracePeriod=30 Aug 04 09:39:30.700849 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.696057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:30.968696 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:30.968661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:31.516320 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:31.516281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:31.602042 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:31.602012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:31.680189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:31.680160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:34.181307 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:34.181267 2562 patch_prober.go:28] interesting pod/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 09:39:34.181681 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:34.181337 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 09:39:35.924009 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:35.923983 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:36.059611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059460 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-token\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059512 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g85hn\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-kube-api-access-g85hn\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059542 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-socket\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059591 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-certs\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059676 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-data\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059744 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-credential-socket\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059778 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d579179-f2c3-40b1-b204-3db8c970d78a-istiod-ca-cert\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059812 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-podinfo\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.059928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.059846 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-envoy\") pod \"3d579179-f2c3-40b1-b204-3db8c970d78a\" (UID: \"3d579179-f2c3-40b1-b204-3db8c970d78a\") " Aug 04 09:39:36.060173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060079 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:39:36.060173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060093 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:39:36.060173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060108 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:39:36.060360 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060180 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.060360 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060197 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.060360 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060284 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:39:36.060528 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.060371 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d579179-f2c3-40b1-b204-3db8c970d78a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:39:36.062630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.062586 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:39:36.062798 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.062692 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:39:36.062798 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.062712 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-kube-api-access-g85hn" (OuterVolumeSpecName: "kube-api-access-g85hn") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "kube-api-access-g85hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:39:36.063026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.063002 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d579179-f2c3-40b1-b204-3db8c970d78a" (UID: "3d579179-f2c3-40b1-b204-3db8c970d78a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:39:36.160815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160785 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.160815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160814 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d579179-f2c3-40b1-b204-3db8c970d78a-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.160815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160826 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.160815 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160834 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.161084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160841 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.161084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160850 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g85hn\" (UniqueName: \"kubernetes.io/projected/3d579179-f2c3-40b1-b204-3db8c970d78a-kube-api-access-g85hn\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.161084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.160858 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d579179-f2c3-40b1-b204-3db8c970d78a-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:39:36.347916 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.347825 2562 generic.go:356] "Generic (PLEG): container finished" podID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerID="a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8" exitCode=0 Aug 04 09:39:36.347916 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.347894 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" Aug 04 09:39:36.348102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.347893 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" event={"ID":"3d579179-f2c3-40b1-b204-3db8c970d78a","Type":"ContainerDied","Data":"a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8"} Aug 04 09:39:36.348102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.348033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js" event={"ID":"3d579179-f2c3-40b1-b204-3db8c970d78a","Type":"ContainerDied","Data":"7f00484620745b5316b929b4d472464ffd9996a9627870e35a7e6acf4d8ffa58"} Aug 04 09:39:36.348102 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.348068 2562 scope.go:117] "RemoveContainer" containerID="a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8" Aug 04 09:39:36.361388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.358794 2562 scope.go:117] "RemoveContainer" containerID="4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87" Aug 04 09:39:36.374904 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.374883 2562 scope.go:117] "RemoveContainer" containerID="a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8" Aug 04 09:39:36.375029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.375009 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js"] Aug 04 09:39:36.375175 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:39:36.375157 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8\": container with ID starting with a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8 not found: ID does not exist" containerID="a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8" Aug 04 09:39:36.375235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.375181 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8"} err="failed to get container status \"a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8\": rpc error: code = NotFound desc = could not find container \"a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8\": container with ID starting with a338196cae993df35b3fa9927898ada2d779d066bbdc3c4b8a7e90a273bc2aa8 not found: ID does not exist" Aug 04 09:39:36.375235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.375199 2562 scope.go:117] "RemoveContainer" containerID="4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87" Aug 04 09:39:36.375457 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:39:36.375436 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87\": container with ID starting with 4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87 not found: ID does not exist" containerID="4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87" Aug 04 09:39:36.375519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.375467 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87"} err="failed to get container status \"4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87\": rpc error: code = NotFound desc = could not find container \"4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87\": container with ID starting with 4a67f27ee053f45868721387ff93b17e0fda23a74df6e1a926efe836c43c5c87 not found: ID does not exist" Aug 04 09:39:36.386968 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:36.386940 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-cf8d7038-openshift-default-6f4b4c947b-mt5js"] Aug 04 09:39:37.417443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:37.417402 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" path="/var/lib/kubelet/pods/3d579179-f2c3-40b1-b204-3db8c970d78a/volumes" Aug 04 09:39:43.727008 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:43.726973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:50.217855 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:50.217816 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:50.317735 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:50.317695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:50.422949 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:50.422911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:50.526296 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:50.526186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:50.621899 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:50.621865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:39:51.152005 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:39:51.151972 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:17.371136 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:17.371092 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:23.365695 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:23.365661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:23.475933 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:23.475893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:23.567331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:23.567284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:39.581532 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:39.581497 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:45.366680 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:45.366630 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:45.482326 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:45.482297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:45.571836 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:45.571804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:56.195818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:40:56.195743 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:07.391355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:07.391325 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:13.620962 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:13.620924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:15.358846 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:15.358801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:16.911079 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:16.911045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:22.658336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:22.658304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:22.719149 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:22.719120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:23.397115 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:23.397084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:29.300898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.300863 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:29.401409 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.401382 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:29.600387 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.600307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:29.890359 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.890271 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv"] Aug 04 09:41:29.890793 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.890769 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" Aug 04 09:41:29.890793 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.890793 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" Aug 04 09:41:29.891008 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.890923 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" Aug 04 09:41:29.893936 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.893913 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:29.896219 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.896199 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ea2eafd7-openshift-default-dockercfg-jqc9q\"" Aug 04 09:41:29.921036 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.921003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv"] Aug 04 09:41:29.932634 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:29.932605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:30.070058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-podinfo\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-token\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-certs\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070175 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istiod-ca-cert\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070215 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshld\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-kube-api-access-sshld\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-credential-socket\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070459 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-envoy\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070459 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-socket\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.070459 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.070346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-data\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.170967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.170940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-socket\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.170974 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-data\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-podinfo\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-token\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171088 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-certs\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171137 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istiod-ca-cert\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171163 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sshld\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-kube-api-access-sshld\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-credential-socket\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171376 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-envoy\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171625 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171535 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-data\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171625 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-socket\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171625 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-credential-socket\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171793 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-certs\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.171863 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.171845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istiod-ca-cert\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.173542 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.173522 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-envoy\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.173631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.173617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-podinfo\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.179068 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.179042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-token\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.179299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.179281 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshld\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-kube-api-access-sshld\") pod \"e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.204433 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.204413 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:30.341125 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.341102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv"] Aug 04 09:41:30.342967 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:41:30.342933 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93de34da_d931_41ba_a4f9_afd3b4abb0b9.slice/crio-87264bdad8691570c585ca9a54e1a0cac6ec845c95b7874f47076148f1068c57 WatchSource:0}: Error finding container 87264bdad8691570c585ca9a54e1a0cac6ec845c95b7874f47076148f1068c57: Status 404 returned error can't find the container with id 87264bdad8691570c585ca9a54e1a0cac6ec845c95b7874f47076148f1068c57 Aug 04 09:41:30.345348 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.345304 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:41:30.345422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.345388 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:41:30.345468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.345430 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:41:30.571975 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.571861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:30.975243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.975210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" event={"ID":"93de34da-d931-41ba-a4f9-afd3b4abb0b9","Type":"ContainerStarted","Data":"3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec"} Aug 04 09:41:30.975429 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.975271 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" event={"ID":"93de34da-d931-41ba-a4f9-afd3b4abb0b9","Type":"ContainerStarted","Data":"87264bdad8691570c585ca9a54e1a0cac6ec845c95b7874f47076148f1068c57"} Aug 04 09:41:30.998720 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:30.998670 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podStartSLOduration=1.998654527 podStartE2EDuration="1.998654527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:30.99702703 +0000 UTC m=+2008.129236562" watchObservedRunningTime="2026-08-04 09:41:30.998654527 +0000 UTC m=+2008.130864058" Aug 04 09:41:31.204818 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:31.204788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:31.209430 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:31.209408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:31.982273 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:31.982218 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:31.983305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:31.983282 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:33.261324 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:33.261247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:33.349507 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:33.349476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:34.396409 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:34.396376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:34.656087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:34.656001 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:34.944477 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:34.944396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:35.087134 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.087099 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh"] Aug 04 09:41:35.087545 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.087531 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" Aug 04 09:41:35.087591 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.087546 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" Aug 04 09:41:35.087627 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.087614 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d579179-f2c3-40b1-b204-3db8c970d78a" containerName="istio-proxy" Aug 04 09:41:35.090768 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.090752 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.093327 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.093305 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ea2eafd7-dockercfg-cwrsv\"" Aug 04 09:41:35.093463 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.093347 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ea2eafd7\"" Aug 04 09:41:35.105291 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.105271 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh"] Aug 04 09:41:35.127091 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.127069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:35.217537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.217451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9mnl\" (UniqueName: \"kubernetes.io/projected/0e20aff4-cd4b-430d-b842-28caca1789e0-kube-api-access-d9mnl\") pod \"payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.217683 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.217603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e20aff4-cd4b-430d-b842-28caca1789e0-plugins-config-volume\") pod \"payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.318215 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.318181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9mnl\" (UniqueName: \"kubernetes.io/projected/0e20aff4-cd4b-430d-b842-28caca1789e0-kube-api-access-d9mnl\") pod \"payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.318400 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.318298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e20aff4-cd4b-430d-b842-28caca1789e0-plugins-config-volume\") pod \"payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.318926 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.318908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e20aff4-cd4b-430d-b842-28caca1789e0-plugins-config-volume\") pod \"payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.326364 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.326334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9mnl\" (UniqueName: \"kubernetes.io/projected/0e20aff4-cd4b-430d-b842-28caca1789e0-kube-api-access-d9mnl\") pod \"payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.401757 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.401718 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:35.547242 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:35.547216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh"] Aug 04 09:41:35.549045 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:41:35.549017 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e20aff4_cd4b_430d_b842_28caca1789e0.slice/crio-6a437ce7324273b4fbb1b1b8b7e46202930593c45791f7d47b3711402f685085 WatchSource:0}: Error finding container 6a437ce7324273b4fbb1b1b8b7e46202930593c45791f7d47b3711402f685085: Status 404 returned error can't find the container with id 6a437ce7324273b4fbb1b1b8b7e46202930593c45791f7d47b3711402f685085 Aug 04 09:41:36.004479 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:36.004443 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" event={"ID":"0e20aff4-cd4b-430d-b842-28caca1789e0","Type":"ContainerStarted","Data":"336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403"} Aug 04 09:41:36.004646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:36.004493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" event={"ID":"0e20aff4-cd4b-430d-b842-28caca1789e0","Type":"ContainerStarted","Data":"6a437ce7324273b4fbb1b1b8b7e46202930593c45791f7d47b3711402f685085"} Aug 04 09:41:36.004646 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:36.004556 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:36.022639 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:36.022593 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" podStartSLOduration=1.022579557 podStartE2EDuration="1.022579557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:36.020322552 +0000 UTC m=+2013.152532088" watchObservedRunningTime="2026-08-04 09:41:36.022579557 +0000 UTC m=+2013.154789088" Aug 04 09:41:46.983443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:46.983406 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:46.983851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:46.983470 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:47.013420 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.013396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:41:47.074952 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.074921 2562 generic.go:356] "Generic (PLEG): container finished" podID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerID="3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec" exitCode=0 Aug 04 09:41:47.075138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.074996 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" event={"ID":"93de34da-d931-41ba-a4f9-afd3b4abb0b9","Type":"ContainerDied","Data":"3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec"} Aug 04 09:41:47.075468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.075450 2562 scope.go:117] "RemoveContainer" containerID="3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec" Aug 04 09:41:47.076498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.076470 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:41:47.076597 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.076548 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:41:47.076660 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:47.076594 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:41:48.088385 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:48.088348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" event={"ID":"93de34da-d931-41ba-a4f9-afd3b4abb0b9","Type":"ContainerStarted","Data":"0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd"} Aug 04 09:41:48.205270 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:48.205220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:48.205475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:48.205453 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:48.205538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:48.205503 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:49.205676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:49.205644 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:49.206069 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:49.205695 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:50.205535 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:50.205498 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:50.205718 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:50.205556 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:51.205672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:51.205637 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:51.205863 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:51.205691 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:52.205267 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:52.205217 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:52.205449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:52.205292 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:53.205028 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:53.204989 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Aug 04 09:41:53.205438 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:53.205047 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Aug 04 09:41:54.209383 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:54.209347 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:54.209771 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:54.209712 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:54.210439 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:54.210422 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:41:58.230565 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:58.230518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.003844 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:59.003808 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.097992 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:59.097943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.471486 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:41:59.471452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:05.246692 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:05.246647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:05.355480 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:05.355446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:16.129303 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.129230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:16.154796 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.154763 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh"] Aug 04 09:42:16.155017 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.154993 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" podUID="0e20aff4-cd4b-430d-b842-28caca1789e0" containerName="payload-processing" containerID="cri-o://336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403" gracePeriod=30 Aug 04 09:42:16.230345 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.230317 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:16.302125 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.302089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:16.365290 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.365236 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:16.406575 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.406549 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:42:16.499201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.499170 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e20aff4-cd4b-430d-b842-28caca1789e0-plugins-config-volume\") pod \"0e20aff4-cd4b-430d-b842-28caca1789e0\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " Aug 04 09:42:16.499378 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.499285 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9mnl\" (UniqueName: \"kubernetes.io/projected/0e20aff4-cd4b-430d-b842-28caca1789e0-kube-api-access-d9mnl\") pod \"0e20aff4-cd4b-430d-b842-28caca1789e0\" (UID: \"0e20aff4-cd4b-430d-b842-28caca1789e0\") " Aug 04 09:42:16.499638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.499609 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e20aff4-cd4b-430d-b842-28caca1789e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e20aff4-cd4b-430d-b842-28caca1789e0" (UID: "0e20aff4-cd4b-430d-b842-28caca1789e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:42:16.499710 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.499686 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e20aff4-cd4b-430d-b842-28caca1789e0-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:16.501370 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.501347 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e20aff4-cd4b-430d-b842-28caca1789e0-kube-api-access-d9mnl" (OuterVolumeSpecName: "kube-api-access-d9mnl") pod "0e20aff4-cd4b-430d-b842-28caca1789e0" (UID: "0e20aff4-cd4b-430d-b842-28caca1789e0"). InnerVolumeSpecName "kube-api-access-d9mnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:42:16.601112 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.601079 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9mnl\" (UniqueName: \"kubernetes.io/projected/0e20aff4-cd4b-430d-b842-28caca1789e0-kube-api-access-d9mnl\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:16.710525 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:16.710491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:17.252134 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.252100 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e20aff4-cd4b-430d-b842-28caca1789e0" containerID="336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403" exitCode=0 Aug 04 09:42:17.252669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.252144 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" Aug 04 09:42:17.252669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.252182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" event={"ID":"0e20aff4-cd4b-430d-b842-28caca1789e0","Type":"ContainerDied","Data":"336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403"} Aug 04 09:42:17.252669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.252228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh" event={"ID":"0e20aff4-cd4b-430d-b842-28caca1789e0","Type":"ContainerDied","Data":"6a437ce7324273b4fbb1b1b8b7e46202930593c45791f7d47b3711402f685085"} Aug 04 09:42:17.252669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.252270 2562 scope.go:117] "RemoveContainer" containerID="336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403" Aug 04 09:42:17.272031 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.272013 2562 scope.go:117] "RemoveContainer" containerID="336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403" Aug 04 09:42:17.272363 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:42:17.272340 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403\": container with ID starting with 336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403 not found: ID does not exist" containerID="336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403" Aug 04 09:42:17.272435 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.272373 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403"} err="failed to get container status \"336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403\": rpc error: code = NotFound desc = could not find container \"336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403\": container with ID starting with 336f4f96865e6f0f4410984977b5e9aea7e70c103916f1aca07f7ee2bb8be403 not found: ID does not exist" Aug 04 09:42:17.285392 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.285368 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh"] Aug 04 09:42:17.289764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.289741 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ea2eafd7-5dc79ffc4-hztdh"] Aug 04 09:42:17.422202 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:17.422169 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e20aff4-cd4b-430d-b842-28caca1789e0" path="/var/lib/kubelet/pods/0e20aff4-cd4b-430d-b842-28caca1789e0/volumes" Aug 04 09:42:22.519953 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:22.519922 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:22.620273 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:22.620223 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:22.798918 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:22.798839 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv"] Aug 04 09:42:22.799073 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:22.799033 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" containerID="cri-o://0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd" gracePeriod=30 Aug 04 09:42:22.818886 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:22.818859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:23.053706 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:23.053625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:23.182953 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:23.182918 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:23.280622 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:23.280590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:23.419531 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:23.419493 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:24.212238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:24.212198 2562 patch_prober.go:28] interesting pod/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 09:42:24.212611 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:24.212267 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 09:42:28.053179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.053155 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:42:28.211867 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.211838 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istiod-ca-cert\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212024 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.211874 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sshld\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-kube-api-access-sshld\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212024 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.211909 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-socket\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212024 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.211979 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-data\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212044 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-podinfo\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212089 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-certs\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212119 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-token\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212165 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212141 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-envoy\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212186 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-credential-socket\") pod \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\" (UID: \"93de34da-d931-41ba-a4f9-afd3b4abb0b9\") " Aug 04 09:42:28.212422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212309 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:42:28.212422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212341 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:42:28.212422 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212355 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:42:28.212638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212458 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:42:28.212638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212567 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:42:28.212872 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212849 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.212872 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212872 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.213064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212886 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.213064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212900 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.213064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.212913 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.214216 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.214189 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-kube-api-access-sshld" (OuterVolumeSpecName: "kube-api-access-sshld") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "kube-api-access-sshld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:42:28.214333 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.214235 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:42:28.214498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.214480 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:42:28.214589 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.214577 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "93de34da-d931-41ba-a4f9-afd3b4abb0b9" (UID: "93de34da-d931-41ba-a4f9-afd3b4abb0b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:42:28.312630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.312595 2562 generic.go:356] "Generic (PLEG): container finished" podID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerID="0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd" exitCode=0 Aug 04 09:42:28.312630 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.312633 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" event={"ID":"93de34da-d931-41ba-a4f9-afd3b4abb0b9","Type":"ContainerDied","Data":"0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd"} Aug 04 09:42:28.312811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.312644 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" Aug 04 09:42:28.312811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.312672 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv" event={"ID":"93de34da-d931-41ba-a4f9-afd3b4abb0b9","Type":"ContainerDied","Data":"87264bdad8691570c585ca9a54e1a0cac6ec845c95b7874f47076148f1068c57"} Aug 04 09:42:28.312811 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.312694 2562 scope.go:117] "RemoveContainer" containerID="0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd" Aug 04 09:42:28.313373 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.313353 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.313496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.313377 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.313496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.313391 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sshld\" (UniqueName: \"kubernetes.io/projected/93de34da-d931-41ba-a4f9-afd3b4abb0b9-kube-api-access-sshld\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.313496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.313407 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de34da-d931-41ba-a4f9-afd3b4abb0b9-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:42:28.322590 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.322571 2562 scope.go:117] "RemoveContainer" containerID="3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec" Aug 04 09:42:28.330348 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.330329 2562 scope.go:117] "RemoveContainer" containerID="0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd" Aug 04 09:42:28.330584 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:42:28.330564 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd\": container with ID starting with 0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd not found: ID does not exist" containerID="0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd" Aug 04 09:42:28.330660 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.330598 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd"} err="failed to get container status \"0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd\": rpc error: code = NotFound desc = could not find container \"0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd\": container with ID starting with 0a0f37f0298e5c6af3a06ec47a94c91250946a98308e5466d778b615e437b4dd not found: ID does not exist" Aug 04 09:42:28.330660 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.330623 2562 scope.go:117] "RemoveContainer" containerID="3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec" Aug 04 09:42:28.330864 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:42:28.330846 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec\": container with ID starting with 3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec not found: ID does not exist" containerID="3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec" Aug 04 09:42:28.330903 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.330870 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec"} err="failed to get container status \"3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec\": rpc error: code = NotFound desc = could not find container \"3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec\": container with ID starting with 3eaad3a2c239f25be90c7d628d4101cdfdc9e7e94bf180579f493bf8b84424ec not found: ID does not exist" Aug 04 09:42:28.337917 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.337894 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv"] Aug 04 09:42:28.342999 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:28.342979 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ea2eafd7-openshift-default-d9c6c76f6-svqwv"] Aug 04 09:42:29.190310 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:29.190276 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:29.291092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:29.291052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:29.392085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:29.392051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:29.415870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:29.415846 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" path="/var/lib/kubelet/pods/93de34da-d931-41ba-a4f9-afd3b4abb0b9/volumes" Aug 04 09:42:29.497338 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:29.497242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:29.607453 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:29.607423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:35.289756 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:35.289687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:35.390295 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:42:35.390238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:03.582572 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:03.582496 2562 scope.go:117] "RemoveContainer" containerID="12be4cea2a24ab0dc1a58d24c0667bfea243f46b51b6693ca19a7c0cbf7e89d7" Aug 04 09:44:38.589983 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.589946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:38.862716 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.862645 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7"] Aug 04 09:44:38.863040 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863028 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" Aug 04 09:44:38.863084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863043 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" Aug 04 09:44:38.863084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863052 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" Aug 04 09:44:38.863084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863059 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" Aug 04 09:44:38.863181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863084 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e20aff4-cd4b-430d-b842-28caca1789e0" containerName="payload-processing" Aug 04 09:44:38.863181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863090 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e20aff4-cd4b-430d-b842-28caca1789e0" containerName="payload-processing" Aug 04 09:44:38.863181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863151 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" Aug 04 09:44:38.863181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.863158 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e20aff4-cd4b-430d-b842-28caca1789e0" containerName="payload-processing" Aug 04 09:44:38.866273 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.866243 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.868669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.868648 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-18d97d1d-openshift-default-dockercfg-zwhg8\"" Aug 04 09:44:38.880864 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.880842 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7"] Aug 04 09:44:38.945595 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-socket\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.945760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945608 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-certs\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.945760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-data\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.945760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-token\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.945760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28gj5\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-kube-api-access-28gj5\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.945760 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-envoy\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.946006 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-podinfo\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.946006 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4facf071-eaa0-4ff5-a922-3db94f1b8038-istiod-ca-cert\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.946006 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.945867 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-credential-socket\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:38.956089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:38.956063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:39.046870 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-credential-socket\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-socket\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-certs\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-data\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-token\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28gj5\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-kube-api-access-28gj5\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.046990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-envoy\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-podinfo\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4facf071-eaa0-4ff5-a922-3db94f1b8038-istiod-ca-cert\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047309 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-credential-socket\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047367 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-data\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047627 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047423 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-socket\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047627 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-certs\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.047864 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.047845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4facf071-eaa0-4ff5-a922-3db94f1b8038-istiod-ca-cert\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.049524 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.049497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-envoy\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.049623 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.049530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-podinfo\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.055080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.055057 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-token\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.055217 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.055202 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28gj5\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-kube-api-access-28gj5\") pod \"e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.176015 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.175985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:39.316605 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.316574 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7"] Aug 04 09:44:39.318562 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:44:39.318532 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4facf071_eaa0_4ff5_a922_3db94f1b8038.slice/crio-e8e608c9dea07e26ec48c3f9f96c9997b68407d529fa77d2ff37f6763d9b2169 WatchSource:0}: Error finding container e8e608c9dea07e26ec48c3f9f96c9997b68407d529fa77d2ff37f6763d9b2169: Status 404 returned error can't find the container with id e8e608c9dea07e26ec48c3f9f96c9997b68407d529fa77d2ff37f6763d9b2169 Aug 04 09:44:39.320681 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.320651 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:44:39.320759 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.320718 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:44:39.320759 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.320744 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:44:39.480654 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:39.480564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:40.024053 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:40.024013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" event={"ID":"4facf071-eaa0-4ff5-a922-3db94f1b8038","Type":"ContainerStarted","Data":"64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f"} Aug 04 09:44:40.024509 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:40.024073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" event={"ID":"4facf071-eaa0-4ff5-a922-3db94f1b8038","Type":"ContainerStarted","Data":"e8e608c9dea07e26ec48c3f9f96c9997b68407d529fa77d2ff37f6763d9b2169"} Aug 04 09:44:40.044795 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:40.044744 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podStartSLOduration=2.044728053 podStartE2EDuration="2.044728053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:44:40.042998978 +0000 UTC m=+2197.175208511" watchObservedRunningTime="2026-08-04 09:44:40.044728053 +0000 UTC m=+2197.176937586" Aug 04 09:44:40.176750 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:40.176715 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:40.181278 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:40.181235 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:41.029617 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:41.029589 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:41.030727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:41.030708 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:42.037820 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:42.037778 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:42.135833 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:42.135801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:43.303270 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:43.303217 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:43.854449 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:43.854408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:44.154122 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:44.154089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:44.267023 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:44.266983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:57.071440 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:57.071401 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Aug 04 09:44:57.071804 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:57.071477 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Aug 04 09:44:58.119239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:58.119206 2562 generic.go:356] "Generic (PLEG): container finished" podID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerID="64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f" exitCode=0 Aug 04 09:44:58.119661 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:58.119287 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" event={"ID":"4facf071-eaa0-4ff5-a922-3db94f1b8038","Type":"ContainerDied","Data":"64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f"} Aug 04 09:44:58.119707 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:58.119690 2562 scope.go:117] "RemoveContainer" containerID="64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f" Aug 04 09:44:58.120715 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:58.120689 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:44:58.120786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:58.120768 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:44:58.120823 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:58.120815 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:44:59.126178 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:59.126145 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" event={"ID":"4facf071-eaa0-4ff5-a922-3db94f1b8038","Type":"ContainerStarted","Data":"5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860"} Aug 04 09:44:59.176290 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:59.176247 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:44:59.176512 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:59.176490 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 04 09:44:59.176563 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:44:59.176542 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 04 09:45:00.148336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.148303 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p"] Aug 04 09:45:00.149535 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.149511 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="93de34da-d931-41ba-a4f9-afd3b4abb0b9" containerName="istio-proxy" Aug 04 09:45:00.153238 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.153217 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" Aug 04 09:45:00.156118 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.156099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:45:00.167856 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.167835 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" Aug 04 09:45:00.168585 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.168559 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p"] Aug 04 09:45:00.176942 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.176919 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 04 09:45:00.177058 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.176973 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 04 09:45:00.328753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:00.328725 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p"] Aug 04 09:45:00.330396 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:45:00.330370 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2260c067_fc66_4ccc_8ed0_189428a437d4.slice/crio-f39c9199a3f87ea05be39c907c118991dba4bc0069d797110974e0b8542d4e63 WatchSource:0}: Error finding container f39c9199a3f87ea05be39c907c118991dba4bc0069d797110974e0b8542d4e63: Status 404 returned error can't find the container with id f39c9199a3f87ea05be39c907c118991dba4bc0069d797110974e0b8542d4e63 Aug 04 09:45:01.138041 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:01.138007 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" event={"ID":"2260c067-fc66-4ccc-8ed0-189428a437d4","Type":"ContainerStarted","Data":"a54ad79730ddc815a9cb2e2ae24ea45ce33a431bb8de2e9fd425a7c06993bd4a"} Aug 04 09:45:01.138214 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:01.138060 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" event={"ID":"2260c067-fc66-4ccc-8ed0-189428a437d4","Type":"ContainerStarted","Data":"f39c9199a3f87ea05be39c907c118991dba4bc0069d797110974e0b8542d4e63"} Aug 04 09:45:01.177099 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:01.177065 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 04 09:45:01.177498 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:01.177116 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 04 09:45:02.155796 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:02.155751 2562 generic.go:356] "Generic (PLEG): container finished" podID="2260c067-fc66-4ccc-8ed0-189428a437d4" containerID="a54ad79730ddc815a9cb2e2ae24ea45ce33a431bb8de2e9fd425a7c06993bd4a" exitCode=0 Aug 04 09:45:02.156002 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:02.155816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" event={"ID":"2260c067-fc66-4ccc-8ed0-189428a437d4","Type":"ContainerDied","Data":"a54ad79730ddc815a9cb2e2ae24ea45ce33a431bb8de2e9fd425a7c06993bd4a"} Aug 04 09:45:02.176469 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:02.176432 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 04 09:45:02.176597 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:02.176484 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 04 09:45:03.177132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:03.177100 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 04 09:45:03.177506 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:03.177154 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 04 09:45:03.324702 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:03.324673 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" Aug 04 09:45:03.603415 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:03.603386 2562 scope.go:117] "RemoveContainer" containerID="23810956e600d2c33322ae361fd3d8c66b3af86bf6a2df0534d5a65dd2c476d2" Aug 04 09:45:04.172070 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:04.172041 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" Aug 04 09:45:04.172212 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:04.172069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763945-ctt5p" event={"ID":"2260c067-fc66-4ccc-8ed0-189428a437d4","Type":"ContainerDied","Data":"f39c9199a3f87ea05be39c907c118991dba4bc0069d797110974e0b8542d4e63"} Aug 04 09:45:04.172212 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:04.172120 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f39c9199a3f87ea05be39c907c118991dba4bc0069d797110974e0b8542d4e63" Aug 04 09:45:04.180085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:04.180062 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:45:04.180508 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:04.180340 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:45:04.181069 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:04.181053 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:45:07.060725 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:07.060689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:09.584396 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:09.584364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:10.244134 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:10.244094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:23.480643 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:23.480614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:35.918208 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:35.918173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:52.028859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.028823 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps"] Aug 04 09:45:52.029344 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.029262 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2260c067-fc66-4ccc-8ed0-189428a437d4" containerName="cleanup" Aug 04 09:45:52.029344 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.029275 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2260c067-fc66-4ccc-8ed0-189428a437d4" containerName="cleanup" Aug 04 09:45:52.029344 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.029346 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2260c067-fc66-4ccc-8ed0-189428a437d4" containerName="cleanup" Aug 04 09:45:52.032545 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.032529 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:52.034992 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.034961 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61aa84e-b527-4500-93a6-3cf843793a2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps\" (UID: \"d61aa84e-b527-4500-93a6-3cf843793a2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:52.035928 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.035908 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:45:52.036035 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.035972 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 09:45:52.062229 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.062205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps"] Aug 04 09:45:52.135587 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.135552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61aa84e-b527-4500-93a6-3cf843793a2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps\" (UID: \"d61aa84e-b527-4500-93a6-3cf843793a2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:52.136117 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.136099 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61aa84e-b527-4500-93a6-3cf843793a2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps\" (UID: \"d61aa84e-b527-4500-93a6-3cf843793a2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:52.342978 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.342895 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:52.474943 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:52.474916 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps"] Aug 04 09:45:52.476641 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:45:52.476614 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61aa84e_b527_4500_93a6_3cf843793a2b.slice/crio-b9efa45c9eb7956d12ef5cc0c20f7fd16a831322e3286e74b4f269fd5c4d757f WatchSource:0}: Error finding container b9efa45c9eb7956d12ef5cc0c20f7fd16a831322e3286e74b4f269fd5c4d757f: Status 404 returned error can't find the container with id b9efa45c9eb7956d12ef5cc0c20f7fd16a831322e3286e74b4f269fd5c4d757f Aug 04 09:45:53.440568 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:53.440535 2562 generic.go:356] "Generic (PLEG): container finished" podID="d61aa84e-b527-4500-93a6-3cf843793a2b" containerID="4d6359652b9e7ccc978840c95748fe41ccaf4259503c08575c346d5fe79d7e1b" exitCode=0 Aug 04 09:45:53.440959 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:53.440574 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" event={"ID":"d61aa84e-b527-4500-93a6-3cf843793a2b","Type":"ContainerDied","Data":"4d6359652b9e7ccc978840c95748fe41ccaf4259503c08575c346d5fe79d7e1b"} Aug 04 09:45:53.440959 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:53.440620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" event={"ID":"d61aa84e-b527-4500-93a6-3cf843793a2b","Type":"ContainerStarted","Data":"b9efa45c9eb7956d12ef5cc0c20f7fd16a831322e3286e74b4f269fd5c4d757f"} Aug 04 09:45:54.571748 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:54.571724 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:54.652765 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:54.652736 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61aa84e-b527-4500-93a6-3cf843793a2b-maas-api-service-ca\") pod \"d61aa84e-b527-4500-93a6-3cf843793a2b\" (UID: \"d61aa84e-b527-4500-93a6-3cf843793a2b\") " Aug 04 09:45:54.653066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:54.653046 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d61aa84e-b527-4500-93a6-3cf843793a2b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d61aa84e-b527-4500-93a6-3cf843793a2b" (UID: "d61aa84e-b527-4500-93a6-3cf843793a2b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:45:54.753531 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:54.753450 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d61aa84e-b527-4500-93a6-3cf843793a2b-maas-api-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:45:55.452853 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:55.452829 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" Aug 04 09:45:55.452853 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:55.452844 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps" event={"ID":"d61aa84e-b527-4500-93a6-3cf843793a2b","Type":"ContainerDied","Data":"b9efa45c9eb7956d12ef5cc0c20f7fd16a831322e3286e74b4f269fd5c4d757f"} Aug 04 09:45:55.453046 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:45:55.452886 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9efa45c9eb7956d12ef5cc0c20f7fd16a831322e3286e74b4f269fd5c4d757f" Aug 04 09:46:06.297547 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:06.297507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:07.569155 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:07.569125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:07.699909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:07.699869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:07.808089 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:07.808060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:07.888887 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:07.888808 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps"] Aug 04 09:46:07.894665 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:07.894625 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18d97d1d-lxtps"] Aug 04 09:46:07.977373 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:07.977341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:09.415996 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:09.415955 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d61aa84e-b527-4500-93a6-3cf843793a2b" path="/var/lib/kubelet/pods/d61aa84e-b527-4500-93a6-3cf843793a2b/volumes" Aug 04 09:46:11.210816 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:11.210779 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:16.281477 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:16.281440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:39.810576 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:39.810539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:44.695759 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:44.695722 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:44.763809 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:44.763775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:52.417462 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:52.417427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:53.697367 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:53.697332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:59.992199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:46:59.992120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:00.068935 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:00.068888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:00.295764 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:00.295686 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7"] Aug 04 09:47:00.295927 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:00.295903 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" containerID="cri-o://5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860" gracePeriod=30 Aug 04 09:47:00.315418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:00.315383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:00.550797 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:00.550709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:04.183483 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:04.183439 2562 patch_prober.go:28] interesting pod/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 09:47:04.183859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:04.183503 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 09:47:05.538464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.538438 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:47:05.688902 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.688870 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4facf071-eaa0-4ff5-a922-3db94f1b8038-istiod-ca-cert\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.688913 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-envoy\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.688945 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-data\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.688977 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-socket\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.688993 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-credential-socket\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689080 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689019 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-token\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689092 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-certs\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28gj5\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-kube-api-access-28gj5\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689192 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-podinfo\") pod \"4facf071-eaa0-4ff5-a922-3db94f1b8038\" (UID: \"4facf071-eaa0-4ff5-a922-3db94f1b8038\") " Aug 04 09:47:05.689388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689291 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-data" (OuterVolumeSpecName: "istio-data") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:05.689584 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689412 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:05.689584 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689433 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4facf071-eaa0-4ff5-a922-3db94f1b8038-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:47:05.689584 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689441 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:05.689754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689589 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:05.689754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689648 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4facf071-eaa0-4ff5-a922-3db94f1b8038-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.689754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689669 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.689754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689684 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.689754 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.689705 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.691222 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.691201 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-token" (OuterVolumeSpecName: "istio-token") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:05.691372 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.691353 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-kube-api-access-28gj5" (OuterVolumeSpecName: "kube-api-access-28gj5") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "kube-api-access-28gj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:05.691559 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.691540 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:05.691619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.691560 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4facf071-eaa0-4ff5-a922-3db94f1b8038" (UID: "4facf071-eaa0-4ff5-a922-3db94f1b8038"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:47:05.791114 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.791085 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.791114 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.791111 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.791311 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.791126 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28gj5\" (UniqueName: \"kubernetes.io/projected/4facf071-eaa0-4ff5-a922-3db94f1b8038-kube-api-access-28gj5\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.791311 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.791139 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.791311 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.791148 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4facf071-eaa0-4ff5-a922-3db94f1b8038-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:47:05.838888 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.838855 2562 generic.go:356] "Generic (PLEG): container finished" podID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerID="5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860" exitCode=0 Aug 04 09:47:05.839033 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.838904 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" Aug 04 09:47:05.839033 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.838941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" event={"ID":"4facf071-eaa0-4ff5-a922-3db94f1b8038","Type":"ContainerDied","Data":"5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860"} Aug 04 09:47:05.839033 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.838993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7" event={"ID":"4facf071-eaa0-4ff5-a922-3db94f1b8038","Type":"ContainerDied","Data":"e8e608c9dea07e26ec48c3f9f96c9997b68407d529fa77d2ff37f6763d9b2169"} Aug 04 09:47:05.839187 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.839078 2562 scope.go:117] "RemoveContainer" containerID="5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860" Aug 04 09:47:05.848551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.848533 2562 scope.go:117] "RemoveContainer" containerID="64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f" Aug 04 09:47:05.856569 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.856551 2562 scope.go:117] "RemoveContainer" containerID="5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860" Aug 04 09:47:05.856826 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:47:05.856808 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860\": container with ID starting with 5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860 not found: ID does not exist" containerID="5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860" Aug 04 09:47:05.856879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.856834 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860"} err="failed to get container status \"5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860\": rpc error: code = NotFound desc = could not find container \"5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860\": container with ID starting with 5c5be7589aff494da1ab80eb7e6525e9e9da46164b20d1e7ea5fb20c68be6860 not found: ID does not exist" Aug 04 09:47:05.856879 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.856853 2562 scope.go:117] "RemoveContainer" containerID="64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f" Aug 04 09:47:05.857052 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:47:05.857030 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f\": container with ID starting with 64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f not found: ID does not exist" containerID="64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f" Aug 04 09:47:05.857093 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.857059 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f"} err="failed to get container status \"64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f\": rpc error: code = NotFound desc = could not find container \"64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f\": container with ID starting with 64df06d6f569dc4ad9ebc3f4ec433c34a7486ea21c631cff280d1646d99e646f not found: ID does not exist" Aug 04 09:47:05.863546 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.863527 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7"] Aug 04 09:47:05.868771 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:05.868750 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-18d97d1d-openshift-default-75cb5fc984-gmkb7"] Aug 04 09:47:07.416336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:07.416301 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" path="/var/lib/kubelet/pods/4facf071-eaa0-4ff5-a922-3db94f1b8038/volumes" Aug 04 09:47:11.875452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:11.875413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:11.991237 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:11.991206 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:12.091573 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:12.091542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:12.762732 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:12.762692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:12.868982 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:12.868942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:12.965876 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:12.965826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:14.294609 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:14.294560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:21.066766 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:21.066709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:21.179989 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:21.179951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:21.800022 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:21.799988 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:28.472338 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:28.472299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:28.582308 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:28.582262 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:28.716196 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:28.716166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:39.256895 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:39.256863 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:39.510298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:47:39.510199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:00.500514 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:00.500477 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:07.084126 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.084089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:07.182587 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.182552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:07.472360 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.472323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:07.788146 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788075 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6"] Aug 04 09:48:07.788471 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788458 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" Aug 04 09:48:07.788519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788472 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" Aug 04 09:48:07.788519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788482 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" Aug 04 09:48:07.788519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788487 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" Aug 04 09:48:07.788519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788497 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d61aa84e-b527-4500-93a6-3cf843793a2b" containerName="revoke-keys" Aug 04 09:48:07.788519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788502 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d61aa84e-b527-4500-93a6-3cf843793a2b" containerName="revoke-keys" Aug 04 09:48:07.788676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788556 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" Aug 04 09:48:07.788676 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.788567 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d61aa84e-b527-4500-93a6-3cf843793a2b" containerName="revoke-keys" Aug 04 09:48:07.797613 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.797584 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.800329 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.800306 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4d301b-openshift-default-dockercfg-k9xs2\"" Aug 04 09:48:07.807012 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.806991 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6"] Aug 04 09:48:07.832816 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.832789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:07.942201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-certs\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-envoy\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c9af47-8cea-4fd7-953b-3262360f1a19-istiod-ca-cert\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942292 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-data\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-socket\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-credential-socket\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xw6c\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-kube-api-access-2xw6c\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-podinfo\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.942452 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.942406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-token\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:07.957108 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:07.957078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:08.042963 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.042872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-data\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.042963 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.042934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-socket\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-credential-socket\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xw6c\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-kube-api-access-2xw6c\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043112 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-podinfo\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-token\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-certs\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043338 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-socket\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043384 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-data\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-credential-socket\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-envoy\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043504 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c9af47-8cea-4fd7-953b-3262360f1a19-istiod-ca-cert\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.043621 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.043539 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-certs\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.044094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.044071 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c9af47-8cea-4fd7-953b-3262360f1a19-istiod-ca-cert\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.045522 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.045494 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-envoy\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.045614 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.045547 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-podinfo\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.052066 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.052044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-token\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.052304 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.052286 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xw6c\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-kube-api-access-2xw6c\") pod \"e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.109275 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.109237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:08.253933 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.253909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6"] Aug 04 09:48:08.256043 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:48:08.256014 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c9af47_8cea_4fd7_953b_3262360f1a19.slice/crio-cf7a3fb7a7b7a073f2fffdcfb524d38f5acb728bf44da1bbf986e176b595306a WatchSource:0}: Error finding container cf7a3fb7a7b7a073f2fffdcfb524d38f5acb728bf44da1bbf986e176b595306a: Status 404 returned error can't find the container with id cf7a3fb7a7b7a073f2fffdcfb524d38f5acb728bf44da1bbf986e176b595306a Aug 04 09:48:08.258384 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.258346 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:48:08.258564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.258499 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:48:08.258564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:08.258552 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:48:09.195359 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:09.195321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" event={"ID":"a5c9af47-8cea-4fd7-953b-3262360f1a19","Type":"ContainerStarted","Data":"c378cc7d848e982c6f007c6ea68085db86ceee1be33088803e6d6dca917dd5d5"} Aug 04 09:48:09.195741 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:09.195383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" event={"ID":"a5c9af47-8cea-4fd7-953b-3262360f1a19","Type":"ContainerStarted","Data":"cf7a3fb7a7b7a073f2fffdcfb524d38f5acb728bf44da1bbf986e176b595306a"} Aug 04 09:48:09.222394 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:09.222346 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podStartSLOduration=2.222330593 podStartE2EDuration="2.222330593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:09.220208751 +0000 UTC m=+2406.352418283" watchObservedRunningTime="2026-08-04 09:48:09.222330593 +0000 UTC m=+2406.354540124" Aug 04 09:48:10.110377 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:10.110337 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:10.115087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:10.115062 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:10.201714 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:10.201679 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:10.202714 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:10.202695 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:11.195245 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:11.195212 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:11.293294 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:11.293236 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:12.331996 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:12.331964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:12.610478 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:12.610391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:13.251236 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.251194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:13.417595 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.417535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:13.494566 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.494534 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf"] Aug 04 09:48:13.495153 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.495136 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4facf071-eaa0-4ff5-a922-3db94f1b8038" containerName="istio-proxy" Aug 04 09:48:13.497315 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.497292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.506937 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.506910 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-4d301b\"" Aug 04 09:48:13.507198 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.507175 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-4d301b-dockercfg-p9qq5\"" Aug 04 09:48:13.517519 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.517496 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf"] Aug 04 09:48:13.595623 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.595596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.595783 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.595648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvxkr\" (UniqueName: \"kubernetes.io/projected/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-kube-api-access-zvxkr\") pod \"payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.696209 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.696179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvxkr\" (UniqueName: \"kubernetes.io/projected/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-kube-api-access-zvxkr\") pod \"payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.696406 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.696297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.696921 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.696902 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.710710 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.710689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvxkr\" (UniqueName: \"kubernetes.io/projected/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-kube-api-access-zvxkr\") pod \"payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.808648 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.808570 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:13.951072 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:13.951041 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf"] Aug 04 09:48:13.952687 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:48:13.952660 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38403ea1_9740_48d6_b992_0f1e5cbb1ad3.slice/crio-a6bdf2e992e941ef39ef2d882f77c386e2602802ab56a5440e8b701aec3c5db8 WatchSource:0}: Error finding container a6bdf2e992e941ef39ef2d882f77c386e2602802ab56a5440e8b701aec3c5db8: Status 404 returned error can't find the container with id a6bdf2e992e941ef39ef2d882f77c386e2602802ab56a5440e8b701aec3c5db8 Aug 04 09:48:14.231355 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:14.231316 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" event={"ID":"38403ea1-9740-48d6-b992-0f1e5cbb1ad3","Type":"ContainerStarted","Data":"77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21"} Aug 04 09:48:14.231527 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:14.231370 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" event={"ID":"38403ea1-9740-48d6-b992-0f1e5cbb1ad3","Type":"ContainerStarted","Data":"a6bdf2e992e941ef39ef2d882f77c386e2602802ab56a5440e8b701aec3c5db8"} Aug 04 09:48:14.231527 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:14.231413 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:14.265482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:14.265428 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" podStartSLOduration=1.265410436 podStartE2EDuration="1.265410436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:14.262218121 +0000 UTC m=+2411.394427653" watchObservedRunningTime="2026-08-04 09:48:14.265410436 +0000 UTC m=+2411.397619969" Aug 04 09:48:25.203361 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.203243 2562 patch_prober.go:28] interesting pod/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 04 09:48:25.203361 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.203340 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 04 09:48:25.239769 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.239738 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:48:25.303228 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.303196 2562 generic.go:356] "Generic (PLEG): container finished" podID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerID="c378cc7d848e982c6f007c6ea68085db86ceee1be33088803e6d6dca917dd5d5" exitCode=0 Aug 04 09:48:25.303434 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.303282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" event={"ID":"a5c9af47-8cea-4fd7-953b-3262360f1a19","Type":"ContainerDied","Data":"c378cc7d848e982c6f007c6ea68085db86ceee1be33088803e6d6dca917dd5d5"} Aug 04 09:48:25.303707 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.303692 2562 scope.go:117] "RemoveContainer" containerID="c378cc7d848e982c6f007c6ea68085db86ceee1be33088803e6d6dca917dd5d5" Aug 04 09:48:25.304672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.304647 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:48:25.304743 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.304709 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:48:25.304800 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:25.304741 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:48:26.310692 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:26.310660 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" event={"ID":"a5c9af47-8cea-4fd7-953b-3262360f1a19","Type":"ContainerStarted","Data":"3da7fc66c0885acc93110cca46887937480355f84db0b1b63b506ca55e089ce7"} Aug 04 09:48:27.110186 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:27.110150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:27.110399 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:27.110379 2562 patch_prober.go:28] interesting pod/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 04 09:48:27.110455 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:27.110422 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 04 09:48:28.110550 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:28.110511 2562 patch_prober.go:28] interesting pod/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 04 09:48:28.110935 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:28.110577 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 04 09:48:29.109739 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:29.109702 2562 patch_prober.go:28] interesting pod/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 04 09:48:29.109893 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:29.109761 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 04 09:48:30.109838 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:30.109804 2562 patch_prober.go:28] interesting pod/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 04 09:48:30.110229 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:30.109861 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 04 09:48:31.110363 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:31.110321 2562 patch_prober.go:28] interesting pod/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 04 09:48:31.110863 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:31.110388 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 04 09:48:32.113520 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:32.113489 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:32.114001 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:32.113782 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:32.114505 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:32.114488 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:48:36.242490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:36.242458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:36.598810 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:36.598732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:36.980619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:36.980585 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:37.373618 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:37.373523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:40.499898 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:40.499859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:40.605934 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:40.605897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:41.448464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:41.448434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:41.555730 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:41.555696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:42.053263 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.053211 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:42.208950 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.208917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:42.277293 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.277240 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd"] Aug 04 09:48:42.280695 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.280679 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.284067 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.284047 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-406b0d\"" Aug 04 09:48:42.300856 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.300835 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd"] Aug 04 09:48:42.373545 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.373475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae360b4-d0b7-482a-b125-751ff01c1143-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.373545 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.373520 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcxdf\" (UniqueName: \"kubernetes.io/projected/5ae360b4-d0b7-482a-b125-751ff01c1143-kube-api-access-tcxdf\") pod \"payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.474693 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.474659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcxdf\" (UniqueName: \"kubernetes.io/projected/5ae360b4-d0b7-482a-b125-751ff01c1143-kube-api-access-tcxdf\") pod \"payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.474859 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.474753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae360b4-d0b7-482a-b125-751ff01c1143-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.475424 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.475404 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae360b4-d0b7-482a-b125-751ff01c1143-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.487536 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.487508 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcxdf\" (UniqueName: \"kubernetes.io/projected/5ae360b4-d0b7-482a-b125-751ff01c1143-kube-api-access-tcxdf\") pod \"payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.590586 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.590550 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:42.735946 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:42.735703 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd"] Aug 04 09:48:43.415425 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:43.415394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" event={"ID":"5ae360b4-d0b7-482a-b125-751ff01c1143","Type":"ContainerStarted","Data":"552dbd1c0a1b550f664193f8a74391354cccc3c2cb2a646da11989ead4d8f4d0"} Aug 04 09:48:43.415425 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:43.415427 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" event={"ID":"5ae360b4-d0b7-482a-b125-751ff01c1143","Type":"ContainerStarted","Data":"f287e2fc0cb875e071ba080456071101f632008f70907f920a8576c9cab3f554"} Aug 04 09:48:43.415643 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:43.415448 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:48:43.436966 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:43.436916 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" podStartSLOduration=1.436903549 podStartE2EDuration="1.436903549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:43.435890661 +0000 UTC m=+2440.568100194" watchObservedRunningTime="2026-08-04 09:48:43.436903549 +0000 UTC m=+2440.569113137" Aug 04 09:48:54.418880 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:48:54.418850 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:49:05.297235 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:05.297201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:24.365344 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:24.365307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:47.656996 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.656962 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj"] Aug 04 09:49:47.660420 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.660403 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.663081 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.663060 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-406b0d\"/\"openshift-service-ca.crt\"" Aug 04 09:49:47.664107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.664088 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-406b0d\"/\"kube-root-ca.crt\"" Aug 04 09:49:47.664107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.664099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-406b0d\"/\"test-model-406b0d-kserve-self-signed-certs\"" Aug 04 09:49:47.664288 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.664134 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-406b0d\"/\"default-dockercfg-8628x\"" Aug 04 09:49:47.674466 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.674442 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj"] Aug 04 09:49:47.743059 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.743022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:47.768246 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.768218 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-dshm\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.768372 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.768271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tls-certs\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.768372 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.768338 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-model-cache\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.768450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.768434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-home\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.768488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.768459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tmp-dir\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.768488 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.768483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8hzp\" (UniqueName: \"kubernetes.io/projected/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-kube-api-access-l8hzp\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.868992 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.868959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-home\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.868992 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.868992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tmp-dir\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.869189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.869012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8hzp\" (UniqueName: \"kubernetes.io/projected/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-kube-api-access-l8hzp\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.869189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.869080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-dshm\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.869189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.869106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tls-certs\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.869189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.869143 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-model-cache\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.869462 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.869439 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-home\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.869967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.869495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tmp-dir\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.870118 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.870083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-model-cache\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.872770 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.872748 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tls-certs\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.877411 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.877387 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-dshm\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.879195 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.879173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8hzp\" (UniqueName: \"kubernetes.io/projected/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-kube-api-access-l8hzp\") pod \"test-model-406b0d-kserve-5d7d9698d9-6dttj\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:47.970299 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:47.970193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:48.105848 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:48.105784 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj"] Aug 04 09:49:48.108154 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:49:48.108126 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54e55ba7_d7e2_4c0d_b6d8_1836c689782c.slice/crio-a3728c17394467ca8227ff572172903221a13a0c504b7267dfbf13a4ff90b889 WatchSource:0}: Error finding container a3728c17394467ca8227ff572172903221a13a0c504b7267dfbf13a4ff90b889: Status 404 returned error can't find the container with id a3728c17394467ca8227ff572172903221a13a0c504b7267dfbf13a4ff90b889 Aug 04 09:49:48.780854 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:48.780809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" event={"ID":"54e55ba7-d7e2-4c0d-b6d8-1836c689782c","Type":"ContainerStarted","Data":"4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2"} Aug 04 09:49:48.780854 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:48.780855 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" event={"ID":"54e55ba7-d7e2-4c0d-b6d8-1836c689782c","Type":"ContainerStarted","Data":"a3728c17394467ca8227ff572172903221a13a0c504b7267dfbf13a4ff90b889"} Aug 04 09:49:48.802464 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:48.802418 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" podStartSLOduration=1.8024024600000002 podStartE2EDuration="1.80240246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:49:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:48.799394039 +0000 UTC m=+2505.931603584" watchObservedRunningTime="2026-08-04 09:49:48.80240246 +0000 UTC m=+2505.934611992" Aug 04 09:49:57.971336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:57.971239 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:57.971336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:57.971300 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:57.984031 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:57.983999 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:49:58.848973 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:49:58.848941 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:50:30.410891 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:30.410854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:31.463704 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:31.463670 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj"] Aug 04 09:50:31.555380 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:31.555350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:35.974789 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:35.974753 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:35.982377 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:35.982342 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf"] Aug 04 09:50:35.982610 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:35.982562 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" podUID="38403ea1-9740-48d6-b992-0f1e5cbb1ad3" containerName="payload-processing" containerID="cri-o://77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21" gracePeriod=30 Aug 04 09:50:36.096734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.096703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:36.244692 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.244665 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:50:36.266418 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.266372 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:36.307967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.307940 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvxkr\" (UniqueName: \"kubernetes.io/projected/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-kube-api-access-zvxkr\") pod \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " Aug 04 09:50:36.308136 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.308114 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-plugins-config-volume\") pod \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\" (UID: \"38403ea1-9740-48d6-b992-0f1e5cbb1ad3\") " Aug 04 09:50:36.308467 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.308446 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38403ea1-9740-48d6-b992-0f1e5cbb1ad3" (UID: "38403ea1-9740-48d6-b992-0f1e5cbb1ad3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:50:36.309968 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.309947 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-kube-api-access-zvxkr" (OuterVolumeSpecName: "kube-api-access-zvxkr") pod "38403ea1-9740-48d6-b992-0f1e5cbb1ad3" (UID: "38403ea1-9740-48d6-b992-0f1e5cbb1ad3"). InnerVolumeSpecName "kube-api-access-zvxkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:50:36.409309 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.409277 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:36.409309 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.409305 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvxkr\" (UniqueName: \"kubernetes.io/projected/38403ea1-9740-48d6-b992-0f1e5cbb1ad3-kube-api-access-zvxkr\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:36.728239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:36.728204 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:37.052339 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.052236 2562 generic.go:356] "Generic (PLEG): container finished" podID="38403ea1-9740-48d6-b992-0f1e5cbb1ad3" containerID="77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21" exitCode=0 Aug 04 09:50:37.052339 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.052288 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" Aug 04 09:50:37.052339 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.052322 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" event={"ID":"38403ea1-9740-48d6-b992-0f1e5cbb1ad3","Type":"ContainerDied","Data":"77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21"} Aug 04 09:50:37.052852 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.052375 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf" event={"ID":"38403ea1-9740-48d6-b992-0f1e5cbb1ad3","Type":"ContainerDied","Data":"a6bdf2e992e941ef39ef2d882f77c386e2602802ab56a5440e8b701aec3c5db8"} Aug 04 09:50:37.052852 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.052402 2562 scope.go:117] "RemoveContainer" containerID="77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21" Aug 04 09:50:37.062221 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.062207 2562 scope.go:117] "RemoveContainer" containerID="77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21" Aug 04 09:50:37.062473 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:50:37.062457 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21\": container with ID starting with 77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21 not found: ID does not exist" containerID="77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21" Aug 04 09:50:37.062532 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.062481 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21"} err="failed to get container status \"77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21\": rpc error: code = NotFound desc = could not find container \"77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21\": container with ID starting with 77b0689fba1dd4bde49da3a25bea2186361c0558dcc80212d6985ef3db00ed21 not found: ID does not exist" Aug 04 09:50:37.077366 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.077345 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf"] Aug 04 09:50:37.082522 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.082500 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4d301b-5867c8876b-n6grf"] Aug 04 09:50:37.416971 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:37.416939 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38403ea1-9740-48d6-b992-0f1e5cbb1ad3" path="/var/lib/kubelet/pods/38403ea1-9740-48d6-b992-0f1e5cbb1ad3/volumes" Aug 04 09:50:41.783040 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:41.783008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:46.483231 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.483167 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" podUID="54e55ba7-d7e2-4c0d-b6d8-1836c689782c" containerName="main" containerID="cri-o://4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2" gracePeriod=45 Aug 04 09:50:46.735510 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.735450 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:50:46.807669 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807640 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-home\") pod \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " Aug 04 09:50:46.807819 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807695 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tls-certs\") pod \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " Aug 04 09:50:46.807819 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807728 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-dshm\") pod \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " Aug 04 09:50:46.807819 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807747 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tmp-dir\") pod \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " Aug 04 09:50:46.807819 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807785 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-model-cache\") pod \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " Aug 04 09:50:46.808037 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807888 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8hzp\" (UniqueName: \"kubernetes.io/projected/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-kube-api-access-l8hzp\") pod \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\" (UID: \"54e55ba7-d7e2-4c0d-b6d8-1836c689782c\") " Aug 04 09:50:46.808037 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.807924 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-home" (OuterVolumeSpecName: "home") pod "54e55ba7-d7e2-4c0d-b6d8-1836c689782c" (UID: "54e55ba7-d7e2-4c0d-b6d8-1836c689782c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:46.808127 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.808051 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54e55ba7-d7e2-4c0d-b6d8-1836c689782c" (UID: "54e55ba7-d7e2-4c0d-b6d8-1836c689782c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:46.808127 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.808066 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-model-cache" (OuterVolumeSpecName: "model-cache") pod "54e55ba7-d7e2-4c0d-b6d8-1836c689782c" (UID: "54e55ba7-d7e2-4c0d-b6d8-1836c689782c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:46.808209 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.808192 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-home\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:46.808247 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.808207 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tmp-dir\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:46.808247 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.808226 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-model-cache\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:46.809927 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.809908 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-dshm" (OuterVolumeSpecName: "dshm") pod "54e55ba7-d7e2-4c0d-b6d8-1836c689782c" (UID: "54e55ba7-d7e2-4c0d-b6d8-1836c689782c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:46.810014 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.809997 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-kube-api-access-l8hzp" (OuterVolumeSpecName: "kube-api-access-l8hzp") pod "54e55ba7-d7e2-4c0d-b6d8-1836c689782c" (UID: "54e55ba7-d7e2-4c0d-b6d8-1836c689782c"). InnerVolumeSpecName "kube-api-access-l8hzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:50:46.810054 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.810027 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54e55ba7-d7e2-4c0d-b6d8-1836c689782c" (UID: "54e55ba7-d7e2-4c0d-b6d8-1836c689782c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:50:46.909569 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.909538 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-tls-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:46.909569 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.909564 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-dshm\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:46.909569 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:46.909574 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8hzp\" (UniqueName: \"kubernetes.io/projected/54e55ba7-d7e2-4c0d-b6d8-1836c689782c-kube-api-access-l8hzp\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:47.111011 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.110924 2562 generic.go:356] "Generic (PLEG): container finished" podID="54e55ba7-d7e2-4c0d-b6d8-1836c689782c" containerID="4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2" exitCode=0 Aug 04 09:50:47.111011 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.110970 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" Aug 04 09:50:47.111181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.111005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" event={"ID":"54e55ba7-d7e2-4c0d-b6d8-1836c689782c","Type":"ContainerDied","Data":"4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2"} Aug 04 09:50:47.111181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.111044 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj" event={"ID":"54e55ba7-d7e2-4c0d-b6d8-1836c689782c","Type":"ContainerDied","Data":"a3728c17394467ca8227ff572172903221a13a0c504b7267dfbf13a4ff90b889"} Aug 04 09:50:47.111181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.111063 2562 scope.go:117] "RemoveContainer" containerID="4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2" Aug 04 09:50:47.120963 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.120946 2562 scope.go:117] "RemoveContainer" containerID="4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2" Aug 04 09:50:47.121210 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:50:47.121192 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2\": container with ID starting with 4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2 not found: ID does not exist" containerID="4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2" Aug 04 09:50:47.121297 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.121219 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2"} err="failed to get container status \"4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2\": rpc error: code = NotFound desc = could not find container \"4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2\": container with ID starting with 4c2b56977d7fc56b57f1b6e95af8c2498b3816a9484ce4d2469f17bf04cd01c2 not found: ID does not exist" Aug 04 09:50:47.135189 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.135167 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj"] Aug 04 09:50:47.146311 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.146290 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-406b0d/test-model-406b0d-kserve-5d7d9698d9-6dttj"] Aug 04 09:50:47.416439 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.416411 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54e55ba7-d7e2-4c0d-b6d8-1836c689782c" path="/var/lib/kubelet/pods/54e55ba7-d7e2-4c0d-b6d8-1836c689782c/volumes" Aug 04 09:50:47.647515 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.647481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:47.904915 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.904883 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6"] Aug 04 09:50:47.905149 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.905123 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" containerID="cri-o://3da7fc66c0885acc93110cca46887937480355f84db0b1b63b506ca55e089ce7" gracePeriod=30 Aug 04 09:50:47.933468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:47.933438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:48.216457 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.216371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:48.283964 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.283924 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv"] Aug 04 09:50:48.284414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.284395 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38403ea1-9740-48d6-b992-0f1e5cbb1ad3" containerName="payload-processing" Aug 04 09:50:48.284414 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.284415 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="38403ea1-9740-48d6-b992-0f1e5cbb1ad3" containerName="payload-processing" Aug 04 09:50:48.284583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.284430 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54e55ba7-d7e2-4c0d-b6d8-1836c689782c" containerName="main" Aug 04 09:50:48.284583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.284439 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="54e55ba7-d7e2-4c0d-b6d8-1836c689782c" containerName="main" Aug 04 09:50:48.284583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.284536 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="38403ea1-9740-48d6-b992-0f1e5cbb1ad3" containerName="payload-processing" Aug 04 09:50:48.284583 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.284551 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="54e55ba7-d7e2-4c0d-b6d8-1836c689782c" containerName="main" Aug 04 09:50:48.289030 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.289009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:48.291516 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.291498 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 09:50:48.291615 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.291577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:50:48.296551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.296531 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv"] Aug 04 09:50:48.421475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.421444 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/979d9eae-b3bb-4fad-919f-f4a19c61e221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv\" (UID: \"979d9eae-b3bb-4fad-919f-f4a19c61e221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:48.522425 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.522336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/979d9eae-b3bb-4fad-919f-f4a19c61e221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv\" (UID: \"979d9eae-b3bb-4fad-919f-f4a19c61e221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:48.522968 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.522950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/979d9eae-b3bb-4fad-919f-f4a19c61e221-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv\" (UID: \"979d9eae-b3bb-4fad-919f-f4a19c61e221\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:48.601311 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.601280 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:48.737603 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:48.737580 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv"] Aug 04 09:50:48.739302 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:50:48.739271 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979d9eae_b3bb_4fad_919f_f4a19c61e221.slice/crio-f14b872502b65304ddd2b5b689f0374198494c343b6f846fe7cf592660192465 WatchSource:0}: Error finding container f14b872502b65304ddd2b5b689f0374198494c343b6f846fe7cf592660192465: Status 404 returned error can't find the container with id f14b872502b65304ddd2b5b689f0374198494c343b6f846fe7cf592660192465 Aug 04 09:50:49.124399 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:49.124367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" event={"ID":"979d9eae-b3bb-4fad-919f-f4a19c61e221","Type":"ContainerStarted","Data":"5fb29c18688b222306fd3ff7c12407df0986895f80e4fdd53ebf5edd44660a95"} Aug 04 09:50:49.124538 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:49.124446 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" event={"ID":"979d9eae-b3bb-4fad-919f-f4a19c61e221","Type":"ContainerStarted","Data":"f14b872502b65304ddd2b5b689f0374198494c343b6f846fe7cf592660192465"} Aug 04 09:50:49.149376 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:49.149329 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" podStartSLOduration=1.149315473 podStartE2EDuration="1.149315473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:50:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:50:49.147427357 +0000 UTC m=+2566.279636890" watchObservedRunningTime="2026-08-04 09:50:49.149315473 +0000 UTC m=+2566.281525005" Aug 04 09:50:50.132138 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:50.132103 2562 generic.go:356] "Generic (PLEG): container finished" podID="979d9eae-b3bb-4fad-919f-f4a19c61e221" containerID="5fb29c18688b222306fd3ff7c12407df0986895f80e4fdd53ebf5edd44660a95" exitCode=0 Aug 04 09:50:50.132551 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:50.132191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" event={"ID":"979d9eae-b3bb-4fad-919f-f4a19c61e221","Type":"ContainerDied","Data":"5fb29c18688b222306fd3ff7c12407df0986895f80e4fdd53ebf5edd44660a95"} Aug 04 09:50:50.851050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:50.851013 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:50.909374 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:50.909339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:51.262640 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:51.262617 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:51.347064 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:51.347029 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/979d9eae-b3bb-4fad-919f-f4a19c61e221-maas-api-service-ca\") pod \"979d9eae-b3bb-4fad-919f-f4a19c61e221\" (UID: \"979d9eae-b3bb-4fad-919f-f4a19c61e221\") " Aug 04 09:50:51.347375 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:51.347351 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/979d9eae-b3bb-4fad-919f-f4a19c61e221-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "979d9eae-b3bb-4fad-919f-f4a19c61e221" (UID: "979d9eae-b3bb-4fad-919f-f4a19c61e221"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:50:51.448187 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:51.448160 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/979d9eae-b3bb-4fad-919f-f4a19c61e221-maas-api-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:52.144433 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:52.144405 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" Aug 04 09:50:52.144626 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:52.144403 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv" event={"ID":"979d9eae-b3bb-4fad-919f-f4a19c61e221","Type":"ContainerDied","Data":"f14b872502b65304ddd2b5b689f0374198494c343b6f846fe7cf592660192465"} Aug 04 09:50:52.144626 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:52.144527 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f14b872502b65304ddd2b5b689f0374198494c343b6f846fe7cf592660192465" Aug 04 09:50:53.151983 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.151956 2562 generic.go:356] "Generic (PLEG): container finished" podID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerID="3da7fc66c0885acc93110cca46887937480355f84db0b1b63b506ca55e089ce7" exitCode=0 Aug 04 09:50:53.152314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.152031 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" event={"ID":"a5c9af47-8cea-4fd7-953b-3262360f1a19","Type":"ContainerDied","Data":"3da7fc66c0885acc93110cca46887937480355f84db0b1b63b506ca55e089ce7"} Aug 04 09:50:53.152314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.152085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" event={"ID":"a5c9af47-8cea-4fd7-953b-3262360f1a19","Type":"ContainerDied","Data":"cf7a3fb7a7b7a073f2fffdcfb524d38f5acb728bf44da1bbf986e176b595306a"} Aug 04 09:50:53.152314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.152104 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf7a3fb7a7b7a073f2fffdcfb524d38f5acb728bf44da1bbf986e176b595306a" Aug 04 09:50:53.152314 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.152117 2562 scope.go:117] "RemoveContainer" containerID="c378cc7d848e982c6f007c6ea68085db86ceee1be33088803e6d6dca917dd5d5" Aug 04 09:50:53.153967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.153952 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:50:53.262179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262099 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-data\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262137 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-socket\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262179 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262158 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-podinfo\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262212 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c9af47-8cea-4fd7-953b-3262360f1a19-istiod-ca-cert\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262233 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-certs\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262337 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-credential-socket\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262373 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xw6c\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-kube-api-access-2xw6c\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262389 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262413 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-token\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262443 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:53.262482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262449 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-envoy\") pod \"a5c9af47-8cea-4fd7-953b-3262360f1a19\" (UID: \"a5c9af47-8cea-4fd7-953b-3262360f1a19\") " Aug 04 09:50:53.262874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262701 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c9af47-8cea-4fd7-953b-3262360f1a19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:50:53.262874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262840 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:53.262967 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262950 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.263007 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262974 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.263007 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262988 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c9af47-8cea-4fd7-953b-3262360f1a19-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.263007 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.262998 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:53.263129 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.263002 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.264960 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.264932 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:50:53.265071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.264932 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:50:53.265071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.265017 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:50:53.265071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.265040 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-kube-api-access-2xw6c" (OuterVolumeSpecName: "kube-api-access-2xw6c") pod "a5c9af47-8cea-4fd7-953b-3262360f1a19" (UID: "a5c9af47-8cea-4fd7-953b-3262360f1a19"). InnerVolumeSpecName "kube-api-access-2xw6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:50:53.364378 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.364342 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xw6c\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-kube-api-access-2xw6c\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.364378 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.364372 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.364378 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.364384 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.364605 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.364397 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c9af47-8cea-4fd7-953b-3262360f1a19-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:53.364605 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:53.364409 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c9af47-8cea-4fd7-953b-3262360f1a19-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:54.158964 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:54.158936 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6" Aug 04 09:50:54.192607 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:54.192578 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6"] Aug 04 09:50:54.196074 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:54.196044 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4d301b-openshift-default-58bf7d69c8-k97z6"] Aug 04 09:50:55.416482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:55.416447 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" path="/var/lib/kubelet/pods/a5c9af47-8cea-4fd7-953b-3262360f1a19/volumes" Aug 04 09:50:58.960468 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:58.960430 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:59.002899 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.002866 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd"] Aug 04 09:50:59.003107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.003083 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" podUID="5ae360b4-d0b7-482a-b125-751ff01c1143" containerName="payload-pre-processing" containerID="cri-o://552dbd1c0a1b550f664193f8a74391354cccc3c2cb2a646da11989ead4d8f4d0" gracePeriod=30 Aug 04 09:50:59.077094 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.077059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:59.188594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.188516 2562 generic.go:356] "Generic (PLEG): container finished" podID="5ae360b4-d0b7-482a-b125-751ff01c1143" containerID="552dbd1c0a1b550f664193f8a74391354cccc3c2cb2a646da11989ead4d8f4d0" exitCode=0 Aug 04 09:50:59.188594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.188552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" event={"ID":"5ae360b4-d0b7-482a-b125-751ff01c1143","Type":"ContainerDied","Data":"552dbd1c0a1b550f664193f8a74391354cccc3c2cb2a646da11989ead4d8f4d0"} Aug 04 09:50:59.249392 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.249362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:59.260802 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.260778 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:50:59.391703 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.391668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:59.405492 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.405449 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv"] Aug 04 09:50:59.416401 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.416374 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae360b4-d0b7-482a-b125-751ff01c1143-plugins-config-volume\") pod \"5ae360b4-d0b7-482a-b125-751ff01c1143\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " Aug 04 09:50:59.416544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.416478 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcxdf\" (UniqueName: \"kubernetes.io/projected/5ae360b4-d0b7-482a-b125-751ff01c1143-kube-api-access-tcxdf\") pod \"5ae360b4-d0b7-482a-b125-751ff01c1143\" (UID: \"5ae360b4-d0b7-482a-b125-751ff01c1143\") " Aug 04 09:50:59.416786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.416755 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae360b4-d0b7-482a-b125-751ff01c1143-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ae360b4-d0b7-482a-b125-751ff01c1143" (UID: "5ae360b4-d0b7-482a-b125-751ff01c1143"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:50:59.418544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.418517 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae360b4-d0b7-482a-b125-751ff01c1143-kube-api-access-tcxdf" (OuterVolumeSpecName: "kube-api-access-tcxdf") pod "5ae360b4-d0b7-482a-b125-751ff01c1143" (UID: "5ae360b4-d0b7-482a-b125-751ff01c1143"). InnerVolumeSpecName "kube-api-access-tcxdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:50:59.418906 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.418892 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-406b0d-sb4dv"] Aug 04 09:50:59.517672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.517598 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcxdf\" (UniqueName: \"kubernetes.io/projected/5ae360b4-d0b7-482a-b125-751ff01c1143-kube-api-access-tcxdf\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:59.517672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.517626 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae360b4-d0b7-482a-b125-751ff01c1143-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:50:59.806297 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:50:59.806198 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:00.195901 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:00.195871 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" Aug 04 09:51:00.196302 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:00.195902 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd" event={"ID":"5ae360b4-d0b7-482a-b125-751ff01c1143","Type":"ContainerDied","Data":"f287e2fc0cb875e071ba080456071101f632008f70907f920a8576c9cab3f554"} Aug 04 09:51:00.196302 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:00.195966 2562 scope.go:117] "RemoveContainer" containerID="552dbd1c0a1b550f664193f8a74391354cccc3c2cb2a646da11989ead4d8f4d0" Aug 04 09:51:00.218047 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:00.218017 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd"] Aug 04 09:51:00.226548 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:00.225435 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-406b0d-b96c9fd76-jb8kd"] Aug 04 09:51:01.416550 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:01.416514 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae360b4-d0b7-482a-b125-751ff01c1143" path="/var/lib/kubelet/pods/5ae360b4-d0b7-482a-b125-751ff01c1143/volumes" Aug 04 09:51:01.417027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:01.416961 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="979d9eae-b3bb-4fad-919f-f4a19c61e221" path="/var/lib/kubelet/pods/979d9eae-b3bb-4fad-919f-f4a19c61e221/volumes" Aug 04 09:51:05.952557 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:05.952522 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:06.054183 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:06.054152 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:06.235731 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:06.235643 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:06.467804 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:06.467771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:06.826085 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:06.826052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:07.126791 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.126697 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb"] Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127062 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127073 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127091 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae360b4-d0b7-482a-b125-751ff01c1143" containerName="payload-pre-processing" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127097 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae360b4-d0b7-482a-b125-751ff01c1143" containerName="payload-pre-processing" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127117 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979d9eae-b3bb-4fad-919f-f4a19c61e221" containerName="revoke-keys" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127124 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="979d9eae-b3bb-4fad-919f-f4a19c61e221" containerName="revoke-keys" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127177 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127183 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="979d9eae-b3bb-4fad-919f-f4a19c61e221" containerName="revoke-keys" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127192 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae360b4-d0b7-482a-b125-751ff01c1143" containerName="payload-pre-processing" Aug 04 09:51:07.127197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.127198 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" Aug 04 09:51:07.131824 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.131805 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.135539 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.135520 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d27a51-openshift-default-dockercfg-zwlkx\"" Aug 04 09:51:07.145524 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.145501 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb"] Aug 04 09:51:07.170082 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.170059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:07.183640 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-data\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183750 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183658 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j8g4\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-kube-api-access-7j8g4\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183750 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-socket\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183750 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1f815-23bf-4832-b308-3711e1c945c4-istio-podinfo\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-envoy\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1f815-23bf-4832-b308-3711e1c945c4-istiod-ca-cert\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183794 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-certs\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183844 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-istio-token\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.183909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.183895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-credential-socket\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285300 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-socket\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285306 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1f815-23bf-4832-b308-3711e1c945c4-istio-podinfo\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285339 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-envoy\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1f815-23bf-4832-b308-3711e1c945c4-istiod-ca-cert\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-certs\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-istio-token\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285540 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-credential-socket\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285546 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-data\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j8g4\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-kube-api-access-7j8g4\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285574 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-socket\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285769 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-certs\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.285905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285817 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-credential-socket\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.286132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.285944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-data\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.286132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.286064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1f815-23bf-4832-b308-3711e1c945c4-istiod-ca-cert\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.287727 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.287700 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-envoy\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.287890 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.287873 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1f815-23bf-4832-b308-3711e1c945c4-istio-podinfo\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.294180 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.294149 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-istio-token\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.294298 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.294192 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j8g4\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-kube-api-access-7j8g4\") pod \"e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.366243 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.366193 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:07.441334 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.441306 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:07.576322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.576287 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb"] Aug 04 09:51:07.577836 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:51:07.577804 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f1f815_23bf_4832_b308_3711e1c945c4.slice/crio-79a4fe650c75db235b5d5b4afaf9f53605c49d04a311d5db29e9bda213e15978 WatchSource:0}: Error finding container 79a4fe650c75db235b5d5b4afaf9f53605c49d04a311d5db29e9bda213e15978: Status 404 returned error can't find the container with id 79a4fe650c75db235b5d5b4afaf9f53605c49d04a311d5db29e9bda213e15978 Aug 04 09:51:07.579989 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.579955 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:51:07.580092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.580035 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:51:07.580092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:07.580076 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:51:08.243292 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:08.243233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" event={"ID":"09f1f815-23bf-4832-b308-3711e1c945c4","Type":"ContainerStarted","Data":"95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db"} Aug 04 09:51:08.243696 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:08.243307 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" event={"ID":"09f1f815-23bf-4832-b308-3711e1c945c4","Type":"ContainerStarted","Data":"79a4fe650c75db235b5d5b4afaf9f53605c49d04a311d5db29e9bda213e15978"} Aug 04 09:51:08.268892 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:08.268842 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podStartSLOduration=1.268826144 podStartE2EDuration="1.268826144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:08.267166039 +0000 UTC m=+2585.399375573" watchObservedRunningTime="2026-08-04 09:51:08.268826144 +0000 UTC m=+2585.401035676" Aug 04 09:51:08.441423 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:08.441391 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:08.446444 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:08.446420 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:09.249565 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:09.249533 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:09.250428 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:09.250410 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:10.304173 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:10.304134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:10.396619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:10.396586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:11.432011 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:11.431981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:11.710555 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:11.710469 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:12.000140 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.000057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:12.128013 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.127979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:12.263123 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.263051 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt"] Aug 04 09:51:12.263497 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.263481 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" Aug 04 09:51:12.263550 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.263500 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c9af47-8cea-4fd7-953b-3262360f1a19" containerName="istio-proxy" Aug 04 09:51:12.266638 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.266618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.269398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.269374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d27a51\"" Aug 04 09:51:12.269537 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.269511 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d27a51-dockercfg-d9zh8\"" Aug 04 09:51:12.277107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.277079 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt"] Aug 04 09:51:12.329608 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.329578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.329763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.329651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwb74\" (UniqueName: \"kubernetes.io/projected/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-kube-api-access-nwb74\") pod \"payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.430487 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.430455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwb74\" (UniqueName: \"kubernetes.io/projected/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-kube-api-access-nwb74\") pod \"payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.430634 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.430547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.431277 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.431243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.440441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.440409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwb74\" (UniqueName: \"kubernetes.io/projected/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-kube-api-access-nwb74\") pod \"payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.580875 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.580787 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:12.724607 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:12.724581 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt"] Aug 04 09:51:12.726242 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:51:12.726212 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a10d50d_ee72_4fd3_bea2_aba5d217be7e.slice/crio-903279084b160d6243cd3ed1456890bb0971ba3e19eb8a957d8c7a95f69af4a1 WatchSource:0}: Error finding container 903279084b160d6243cd3ed1456890bb0971ba3e19eb8a957d8c7a95f69af4a1: Status 404 returned error can't find the container with id 903279084b160d6243cd3ed1456890bb0971ba3e19eb8a957d8c7a95f69af4a1 Aug 04 09:51:13.273822 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:13.273786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" event={"ID":"8a10d50d-ee72-4fd3-bea2-aba5d217be7e","Type":"ContainerStarted","Data":"eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef"} Aug 04 09:51:13.274029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:13.273869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" event={"ID":"8a10d50d-ee72-4fd3-bea2-aba5d217be7e","Type":"ContainerStarted","Data":"903279084b160d6243cd3ed1456890bb0971ba3e19eb8a957d8c7a95f69af4a1"} Aug 04 09:51:13.274029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:13.273938 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:13.291897 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:13.291848 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" podStartSLOduration=1.291833306 podStartE2EDuration="1.291833306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:13.2904579 +0000 UTC m=+2590.422667432" watchObservedRunningTime="2026-08-04 09:51:13.291833306 +0000 UTC m=+2590.424042852" Aug 04 09:51:23.335370 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:23.335289 2562 generic.go:356] "Generic (PLEG): container finished" podID="09f1f815-23bf-4832-b308-3711e1c945c4" containerID="95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db" exitCode=0 Aug 04 09:51:23.335370 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:23.335360 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" event={"ID":"09f1f815-23bf-4832-b308-3711e1c945c4","Type":"ContainerDied","Data":"95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db"} Aug 04 09:51:23.335786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:23.335744 2562 scope.go:117] "RemoveContainer" containerID="95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db" Aug 04 09:51:23.336732 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:23.336709 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:51:23.336801 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:23.336773 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:51:23.336850 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:23.336813 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:51:24.250446 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:24.250408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:24.282901 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:24.282872 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:51:24.353579 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:24.352608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" event={"ID":"09f1f815-23bf-4832-b308-3711e1c945c4","Type":"ContainerStarted","Data":"905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83"} Aug 04 09:51:24.460880 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:24.460847 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:24.462084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:24.462053 2562 patch_prober.go:28] interesting pod/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 04 09:51:24.462220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:24.462124 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 04 09:51:25.442491 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:25.442400 2562 patch_prober.go:28] interesting pod/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 04 09:51:25.442491 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:25.442461 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 04 09:51:26.441749 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:26.441707 2562 patch_prober.go:28] interesting pod/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 04 09:51:26.442052 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:26.442022 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 04 09:51:27.460700 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:27.460654 2562 patch_prober.go:28] interesting pod/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 04 09:51:27.461195 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:27.460721 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 04 09:51:28.442490 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:28.442450 2562 patch_prober.go:28] interesting pod/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 04 09:51:28.442699 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:28.442526 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 04 09:51:29.446147 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:29.446118 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:29.446609 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:29.446592 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:29.447166 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:29.447151 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:51:35.327722 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:35.327684 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:36.971220 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:36.971184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:37.503109 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:37.503077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:39.188591 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:39.188553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:39.283756 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:39.283725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:40.306994 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.306957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:40.423801 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.423769 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:40.815096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.815065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:40.920291 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.920238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:40.923639 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.923612 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427"] Aug 04 09:51:40.927938 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.927916 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:40.930346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.930327 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0018e3-dockercfg-spdhx\"" Aug 04 09:51:40.930748 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.930727 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0018e3\"" Aug 04 09:51:40.939312 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:40.939290 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427"] Aug 04 09:51:41.097331 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.097222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qz4f\" (UniqueName: \"kubernetes.io/projected/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-kube-api-access-6qz4f\") pod \"payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.097485 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.097403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.198668 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.198625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.198845 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.198803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qz4f\" (UniqueName: \"kubernetes.io/projected/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-kube-api-access-6qz4f\") pod \"payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.199378 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.199356 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.207487 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.207466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qz4f\" (UniqueName: \"kubernetes.io/projected/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-kube-api-access-6qz4f\") pod \"payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.240119 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.240095 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.374441 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.374415 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427"] Aug 04 09:51:41.375680 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:51:41.375644 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34fa9f2_2e08_4acd_9f1b_1a94d23540fb.slice/crio-1cb39fc872fe066e190fd4f58a4710bf11fc95447d7f755bb1177240d2b6ead0 WatchSource:0}: Error finding container 1cb39fc872fe066e190fd4f58a4710bf11fc95447d7f755bb1177240d2b6ead0: Status 404 returned error can't find the container with id 1cb39fc872fe066e190fd4f58a4710bf11fc95447d7f755bb1177240d2b6ead0 Aug 04 09:51:41.454496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.454462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" event={"ID":"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb","Type":"ContainerStarted","Data":"ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb"} Aug 04 09:51:41.454606 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.454526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" event={"ID":"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb","Type":"ContainerStarted","Data":"1cb39fc872fe066e190fd4f58a4710bf11fc95447d7f755bb1177240d2b6ead0"} Aug 04 09:51:41.454606 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.454561 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:51:41.477336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:41.477288 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" podStartSLOduration=1.4772736069999999 podStartE2EDuration="1.477273607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:41.475281785 +0000 UTC m=+2618.607491320" watchObservedRunningTime="2026-08-04 09:51:41.477273607 +0000 UTC m=+2618.609483141" Aug 04 09:51:52.463635 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:51:52.463605 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:52:03.696168 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:03.696133 2562 scope.go:117] "RemoveContainer" containerID="4d6359652b9e7ccc978840c95748fe41ccaf4259503c08575c346d5fe79d7e1b" Aug 04 09:52:04.238790 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:04.238754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:22.908192 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:22.908152 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:22.944813 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:22.944775 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt"] Aug 04 09:52:22.945132 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:22.945102 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" podUID="8a10d50d-ee72-4fd3-bea2-aba5d217be7e" containerName="payload-processing" containerID="cri-o://eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef" gracePeriod=30 Aug 04 09:52:23.012012 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.011979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:23.109121 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.109086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:23.177956 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.177867 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:23.210900 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.210879 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:52:23.285193 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.285163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-plugins-config-volume\") pod \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " Aug 04 09:52:23.285380 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.285203 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwb74\" (UniqueName: \"kubernetes.io/projected/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-kube-api-access-nwb74\") pod \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\" (UID: \"8a10d50d-ee72-4fd3-bea2-aba5d217be7e\") " Aug 04 09:52:23.285653 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.285629 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a10d50d-ee72-4fd3-bea2-aba5d217be7e" (UID: "8a10d50d-ee72-4fd3-bea2-aba5d217be7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:52:23.287377 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.287354 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-kube-api-access-nwb74" (OuterVolumeSpecName: "kube-api-access-nwb74") pod "8a10d50d-ee72-4fd3-bea2-aba5d217be7e" (UID: "8a10d50d-ee72-4fd3-bea2-aba5d217be7e"). InnerVolumeSpecName "kube-api-access-nwb74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:52:23.386481 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.386455 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwb74\" (UniqueName: \"kubernetes.io/projected/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-kube-api-access-nwb74\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:23.386481 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.386477 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a10d50d-ee72-4fd3-bea2-aba5d217be7e-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:23.518784 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.518708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:23.694835 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.694798 2562 generic.go:356] "Generic (PLEG): container finished" podID="8a10d50d-ee72-4fd3-bea2-aba5d217be7e" containerID="eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef" exitCode=0 Aug 04 09:52:23.695023 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.694851 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" Aug 04 09:52:23.695023 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.694862 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" event={"ID":"8a10d50d-ee72-4fd3-bea2-aba5d217be7e","Type":"ContainerDied","Data":"eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef"} Aug 04 09:52:23.695023 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.694905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt" event={"ID":"8a10d50d-ee72-4fd3-bea2-aba5d217be7e","Type":"ContainerDied","Data":"903279084b160d6243cd3ed1456890bb0971ba3e19eb8a957d8c7a95f69af4a1"} Aug 04 09:52:23.695023 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.694926 2562 scope.go:117] "RemoveContainer" containerID="eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef" Aug 04 09:52:23.704048 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.704030 2562 scope.go:117] "RemoveContainer" containerID="eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef" Aug 04 09:52:23.704532 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:52:23.704493 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef\": container with ID starting with eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef not found: ID does not exist" containerID="eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef" Aug 04 09:52:23.704659 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.704530 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef"} err="failed to get container status \"eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef\": rpc error: code = NotFound desc = could not find container \"eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef\": container with ID starting with eb90d978e883a2eddbf743bd2ebc0943bb48b0f0cf32e14f58caa561839a9fef not found: ID does not exist" Aug 04 09:52:23.712721 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.712700 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt"] Aug 04 09:52:23.716333 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:23.716311 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d27a51-7f4478468d-6pvqt"] Aug 04 09:52:25.416050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:25.416011 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a10d50d-ee72-4fd3-bea2-aba5d217be7e" path="/var/lib/kubelet/pods/8a10d50d-ee72-4fd3-bea2-aba5d217be7e/volumes" Aug 04 09:52:29.431435 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:29.431405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:29.531723 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:29.531691 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:29.756668 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:29.756588 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb"] Aug 04 09:52:29.756821 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:29.756786 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" containerID="cri-o://905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83" gracePeriod=30 Aug 04 09:52:29.775959 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:29.775924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:30.017924 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.017828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:30.121857 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.121822 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8"] Aug 04 09:52:30.122265 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.122239 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a10d50d-ee72-4fd3-bea2-aba5d217be7e" containerName="payload-processing" Aug 04 09:52:30.122322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.122266 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a10d50d-ee72-4fd3-bea2-aba5d217be7e" containerName="payload-processing" Aug 04 09:52:30.122359 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.122322 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a10d50d-ee72-4fd3-bea2-aba5d217be7e" containerName="payload-processing" Aug 04 09:52:30.126426 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.126411 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:30.133651 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.133628 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 09:52:30.133909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.133893 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ftr4r\"" Aug 04 09:52:30.149513 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.149491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8"] Aug 04 09:52:30.251732 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.251695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229fe9dd-037a-457d-96d9-878528ff7ac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0018e3-87qq8\" (UID: \"229fe9dd-037a-457d-96d9-878528ff7ac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:30.352765 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.352687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229fe9dd-037a-457d-96d9-878528ff7ac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0018e3-87qq8\" (UID: \"229fe9dd-037a-457d-96d9-878528ff7ac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:30.353361 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.353341 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229fe9dd-037a-457d-96d9-878528ff7ac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0018e3-87qq8\" (UID: \"229fe9dd-037a-457d-96d9-878528ff7ac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:30.436563 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.436531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:30.576666 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.576639 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8"] Aug 04 09:52:30.578046 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:52:30.578021 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229fe9dd_037a_457d_96d9_878528ff7ac6.slice/crio-e44a77c53b6090e9882d7afe2b5ffdba3bb2baf41b3ec7c63898edf80896ae74 WatchSource:0}: Error finding container e44a77c53b6090e9882d7afe2b5ffdba3bb2baf41b3ec7c63898edf80896ae74: Status 404 returned error can't find the container with id e44a77c53b6090e9882d7afe2b5ffdba3bb2baf41b3ec7c63898edf80896ae74 Aug 04 09:52:30.737336 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:30.737303 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" event={"ID":"229fe9dd-037a-457d-96d9-878528ff7ac6","Type":"ContainerStarted","Data":"e44a77c53b6090e9882d7afe2b5ffdba3bb2baf41b3ec7c63898edf80896ae74"} Aug 04 09:52:31.744746 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:31.744706 2562 generic.go:356] "Generic (PLEG): container finished" podID="229fe9dd-037a-457d-96d9-878528ff7ac6" containerID="e4354f419323098bb56e63649e3614ce822da86ac6d8232d872bc9f130786877" exitCode=0 Aug 04 09:52:31.745116 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:31.744790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" event={"ID":"229fe9dd-037a-457d-96d9-878528ff7ac6","Type":"ContainerDied","Data":"e4354f419323098bb56e63649e3614ce822da86ac6d8232d872bc9f130786877"} Aug 04 09:52:32.879195 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:32.879174 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:32.975835 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:32.975805 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229fe9dd-037a-457d-96d9-878528ff7ac6-maas-api-service-ca\") pod \"229fe9dd-037a-457d-96d9-878528ff7ac6\" (UID: \"229fe9dd-037a-457d-96d9-878528ff7ac6\") " Aug 04 09:52:32.976159 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:32.976138 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229fe9dd-037a-457d-96d9-878528ff7ac6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "229fe9dd-037a-457d-96d9-878528ff7ac6" (UID: "229fe9dd-037a-457d-96d9-878528ff7ac6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:52:33.077372 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:33.077288 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229fe9dd-037a-457d-96d9-878528ff7ac6-maas-api-service-ca\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:33.762689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:33.762597 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" Aug 04 09:52:33.762689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:33.762629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8" event={"ID":"229fe9dd-037a-457d-96d9-878528ff7ac6","Type":"ContainerDied","Data":"e44a77c53b6090e9882d7afe2b5ffdba3bb2baf41b3ec7c63898edf80896ae74"} Aug 04 09:52:33.762689 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:33.762690 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e44a77c53b6090e9882d7afe2b5ffdba3bb2baf41b3ec7c63898edf80896ae74" Aug 04 09:52:35.002741 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.002716 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:52:35.095883 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.095814 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-data\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.095883 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.095861 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-credential-socket\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.095885 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1f815-23bf-4832-b308-3711e1c945c4-istio-podinfo\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.095938 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-certs\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.095963 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-istio-token\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096003 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j8g4\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-kube-api-access-7j8g4\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096027 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-envoy\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096077 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096049 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1f815-23bf-4832-b308-3711e1c945c4-istiod-ca-cert\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096088 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-socket\") pod \"09f1f815-23bf-4832-b308-3711e1c945c4\" (UID: \"09f1f815-23bf-4832-b308-3711e1c945c4\") " Aug 04 09:52:35.096432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096117 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:52:35.096432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096124 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:52:35.096432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096356 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:52:35.096628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096477 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.096628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096496 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.096628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096530 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.096628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096476 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:52:35.096628 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.096515 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09f1f815-23bf-4832-b308-3711e1c945c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:52:35.098140 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.098118 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09f1f815-23bf-4832-b308-3711e1c945c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 09:52:35.098232 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.098187 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:52:35.098737 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.098715 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:52:35.098798 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.098777 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-kube-api-access-7j8g4" (OuterVolumeSpecName: "kube-api-access-7j8g4") pod "09f1f815-23bf-4832-b308-3711e1c945c4" (UID: "09f1f815-23bf-4832-b308-3711e1c945c4"). InnerVolumeSpecName "kube-api-access-7j8g4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:52:35.196957 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.196928 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.196957 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.196953 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1f815-23bf-4832-b308-3711e1c945c4-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.196957 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.196963 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.197135 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.196972 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j8g4\" (UniqueName: \"kubernetes.io/projected/09f1f815-23bf-4832-b308-3711e1c945c4-kube-api-access-7j8g4\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.197135 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.196982 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1f815-23bf-4832-b308-3711e1c945c4-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.197135 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.196990 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1f815-23bf-4832-b308-3711e1c945c4-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:35.776757 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.776667 2562 generic.go:356] "Generic (PLEG): container finished" podID="09f1f815-23bf-4832-b308-3711e1c945c4" containerID="905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83" exitCode=0 Aug 04 09:52:35.776757 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.776714 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" Aug 04 09:52:35.776938 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.776716 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" event={"ID":"09f1f815-23bf-4832-b308-3711e1c945c4","Type":"ContainerDied","Data":"905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83"} Aug 04 09:52:35.776938 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.776843 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb" event={"ID":"09f1f815-23bf-4832-b308-3711e1c945c4","Type":"ContainerDied","Data":"79a4fe650c75db235b5d5b4afaf9f53605c49d04a311d5db29e9bda213e15978"} Aug 04 09:52:35.776938 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.776867 2562 scope.go:117] "RemoveContainer" containerID="905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83" Aug 04 09:52:35.785743 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.785716 2562 scope.go:117] "RemoveContainer" containerID="95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db" Aug 04 09:52:35.793556 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.793539 2562 scope.go:117] "RemoveContainer" containerID="905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83" Aug 04 09:52:35.793821 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:52:35.793802 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83\": container with ID starting with 905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83 not found: ID does not exist" containerID="905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83" Aug 04 09:52:35.793877 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.793829 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83"} err="failed to get container status \"905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83\": rpc error: code = NotFound desc = could not find container \"905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83\": container with ID starting with 905a9b1181ccb198095ce9e50130022a553f747b4db9c1e5759cfc8faaa89c83 not found: ID does not exist" Aug 04 09:52:35.793877 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.793846 2562 scope.go:117] "RemoveContainer" containerID="95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db" Aug 04 09:52:35.794061 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:52:35.794038 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db\": container with ID starting with 95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db not found: ID does not exist" containerID="95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db" Aug 04 09:52:35.794099 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.794067 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db"} err="failed to get container status \"95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db\": rpc error: code = NotFound desc = could not find container \"95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db\": container with ID starting with 95dc79fc19f1f62a5137add959a3b9562eba3d18ec1d8c012cef6469ad02e6db not found: ID does not exist" Aug 04 09:52:35.798181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.798160 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb"] Aug 04 09:52:35.805951 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:35.805917 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d27a51-openshift-default-7d9c55945b-mnkhb"] Aug 04 09:52:37.416403 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:37.416371 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" path="/var/lib/kubelet/pods/09f1f815-23bf-4832-b308-3711e1c945c4/volumes" Aug 04 09:52:45.744753 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.744719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:45.780406 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.780370 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427"] Aug 04 09:52:45.780677 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.780622 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" podUID="e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" containerName="payload-pre-processing" containerID="cri-o://ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb" gracePeriod=30 Aug 04 09:52:45.822419 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.822391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:45.922469 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.922436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:45.939544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.939506 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8"] Aug 04 09:52:45.941655 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:45.941637 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0018e3-87qq8"] Aug 04 09:52:46.021071 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.021016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:46.030704 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.030683 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:52:46.088432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.088408 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qz4f\" (UniqueName: \"kubernetes.io/projected/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-kube-api-access-6qz4f\") pod \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " Aug 04 09:52:46.088559 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.088474 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-plugins-config-volume\") pod \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\" (UID: \"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb\") " Aug 04 09:52:46.088873 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.088852 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" (UID: "e34fa9f2-2e08-4acd-9f1b-1a94d23540fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:52:46.090544 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.090519 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-kube-api-access-6qz4f" (OuterVolumeSpecName: "kube-api-access-6qz4f") pod "e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" (UID: "e34fa9f2-2e08-4acd-9f1b-1a94d23540fb"). InnerVolumeSpecName "kube-api-access-6qz4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:52:46.189905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.189873 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qz4f\" (UniqueName: \"kubernetes.io/projected/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-kube-api-access-6qz4f\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:46.189905 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.189899 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 09:52:46.337097 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.336997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:46.842265 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.842206 2562 generic.go:356] "Generic (PLEG): container finished" podID="e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" containerID="ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb" exitCode=0 Aug 04 09:52:46.842734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.842278 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" Aug 04 09:52:46.842734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.842300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" event={"ID":"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb","Type":"ContainerDied","Data":"ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb"} Aug 04 09:52:46.842734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.842370 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427" event={"ID":"e34fa9f2-2e08-4acd-9f1b-1a94d23540fb","Type":"ContainerDied","Data":"1cb39fc872fe066e190fd4f58a4710bf11fc95447d7f755bb1177240d2b6ead0"} Aug 04 09:52:46.842734 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.842407 2562 scope.go:117] "RemoveContainer" containerID="ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb" Aug 04 09:52:46.854502 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.854482 2562 scope.go:117] "RemoveContainer" containerID="ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb" Aug 04 09:52:46.854763 ip-10-0-136-20 kubenswrapper[2562]: E0804 09:52:46.854746 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb\": container with ID starting with ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb not found: ID does not exist" containerID="ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb" Aug 04 09:52:46.854830 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.854771 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb"} err="failed to get container status \"ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb\": rpc error: code = NotFound desc = could not find container \"ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb\": container with ID starting with ab4f3ade5a8f992d713aecb55d00e30d8a1eb08e637fe54c4713fccdb02e11fb not found: ID does not exist" Aug 04 09:52:46.866891 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.866868 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427"] Aug 04 09:52:46.873780 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:46.873756 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0018e3-7cb59f6cf6-v7427"] Aug 04 09:52:47.416322 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:47.416283 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="229fe9dd-037a-457d-96d9-878528ff7ac6" path="/var/lib/kubelet/pods/229fe9dd-037a-457d-96d9-878528ff7ac6/volumes" Aug 04 09:52:47.416713 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:47.416692 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" path="/var/lib/kubelet/pods/e34fa9f2-2e08-4acd-9f1b-1a94d23540fb/volumes" Aug 04 09:52:52.571619 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:52.571586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:52.718050 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:52.718014 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:52.927013 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:52.926976 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:53.130476 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.130427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:53.479649 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.479616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:53.800513 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.800438 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx"] Aug 04 09:52:53.800974 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.800955 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" containerName="payload-pre-processing" Aug 04 09:52:53.801029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.800978 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" containerName="payload-pre-processing" Aug 04 09:52:53.801029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.800988 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="229fe9dd-037a-457d-96d9-878528ff7ac6" containerName="revoke-keys" Aug 04 09:52:53.801029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.800994 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="229fe9dd-037a-457d-96d9-878528ff7ac6" containerName="revoke-keys" Aug 04 09:52:53.801029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.801005 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" Aug 04 09:52:53.801029 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.801012 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" Aug 04 09:52:53.801170 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.801085 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" Aug 04 09:52:53.801170 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.801094 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" Aug 04 09:52:53.801170 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.801100 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34fa9f2-2e08-4acd-9f1b-1a94d23540fb" containerName="payload-pre-processing" Aug 04 09:52:53.801170 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.801107 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="229fe9dd-037a-457d-96d9-878528ff7ac6" containerName="revoke-keys" Aug 04 09:52:53.805665 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.805648 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.809639 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.809614 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-497679-openshift-default-dockercfg-lqd5s\"" Aug 04 09:52:53.849501 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.849476 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx"] Aug 04 09:52:53.856027 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.856007 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:53.860388 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-certs\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-istio-token\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-socket\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-data\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860482 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c403c916-d126-40ac-83b6-68a84957e9e6-istio-podinfo\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c403c916-d126-40ac-83b6-68a84957e9e6-istiod-ca-cert\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-envoy\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860657 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv2rk\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-kube-api-access-tv2rk\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.860851 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.860723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-credential-socket\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.961873 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.961837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-credential-socket\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.961891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-certs\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.961916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-istio-token\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.961945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-socket\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.961972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-data\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962026 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c403c916-d126-40ac-83b6-68a84957e9e6-istio-podinfo\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962306 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c403c916-d126-40ac-83b6-68a84957e9e6-istiod-ca-cert\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962306 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-envoy\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962306 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv2rk\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-kube-api-access-tv2rk\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962362 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-credential-socket\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962390 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-certs\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962475 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-socket\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962605 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-data\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.962839 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.962821 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c403c916-d126-40ac-83b6-68a84957e9e6-istiod-ca-cert\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.964412 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.964389 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-envoy\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.964594 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.964575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c403c916-d126-40ac-83b6-68a84957e9e6-istio-podinfo\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.970513 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.970491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-istio-token\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:53.970771 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:53.970750 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv2rk\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-kube-api-access-tv2rk\") pod \"e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:54.103681 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.103611 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:54.115228 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.115198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:54.267695 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.267664 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx"] Aug 04 09:52:54.268845 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:52:54.268817 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc403c916_d126_40ac_83b6_68a84957e9e6.slice/crio-e6057098461f741a7ffa9bb0ce867a99f2e346a5d6285f8d91d287d8a8cd34c9 WatchSource:0}: Error finding container e6057098461f741a7ffa9bb0ce867a99f2e346a5d6285f8d91d287d8a8cd34c9: Status 404 returned error can't find the container with id e6057098461f741a7ffa9bb0ce867a99f2e346a5d6285f8d91d287d8a8cd34c9 Aug 04 09:52:54.270858 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.270828 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:52:54.270949 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.270889 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:52:54.270949 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.270915 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:52:54.896163 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.896122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" event={"ID":"c403c916-d126-40ac-83b6-68a84957e9e6","Type":"ContainerStarted","Data":"79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7"} Aug 04 09:52:54.896658 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.896188 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" event={"ID":"c403c916-d126-40ac-83b6-68a84957e9e6","Type":"ContainerStarted","Data":"e6057098461f741a7ffa9bb0ce867a99f2e346a5d6285f8d91d287d8a8cd34c9"} Aug 04 09:52:54.924985 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:54.924938 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podStartSLOduration=1.9249242610000001 podStartE2EDuration="1.924924261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:52:54.92291019 +0000 UTC m=+2692.055119722" watchObservedRunningTime="2026-08-04 09:52:54.924924261 +0000 UTC m=+2692.057133792" Aug 04 09:52:55.116432 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:55.116392 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:55.121201 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:55.121180 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:55.901617 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:55.901584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:55.902450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:55.902433 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:52:56.894923 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:56.894881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:56.984910 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:56.984877 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:58.037496 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.037460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:58.340957 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.340879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:58.649351 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.649324 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:58.764178 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.764143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:58.951564 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.951495 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp"] Aug 04 09:52:58.951893 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.951881 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" Aug 04 09:52:58.951938 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.951896 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f1f815-23bf-4832-b308-3711e1c945c4" containerName="istio-proxy" Aug 04 09:52:58.955199 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.955181 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:58.958659 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.958631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-497679\"" Aug 04 09:52:58.958945 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.958927 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-497679-dockercfg-wkf5h\"" Aug 04 09:52:58.970750 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:58.970725 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp"] Aug 04 09:52:59.005908 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.005883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qkjr\" (UniqueName: \"kubernetes.io/projected/c25bb48f-0b54-422d-9d24-9808f67f8980-kube-api-access-6qkjr\") pod \"payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.006054 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.005951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c25bb48f-0b54-422d-9d24-9808f67f8980-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.106907 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.106877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qkjr\" (UniqueName: \"kubernetes.io/projected/c25bb48f-0b54-422d-9d24-9808f67f8980-kube-api-access-6qkjr\") pod \"payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.107305 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.106940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c25bb48f-0b54-422d-9d24-9808f67f8980-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.107577 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.107559 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c25bb48f-0b54-422d-9d24-9808f67f8980-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.120087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.120061 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qkjr\" (UniqueName: \"kubernetes.io/projected/c25bb48f-0b54-422d-9d24-9808f67f8980-kube-api-access-6qkjr\") pod \"payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.266346 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.266238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.404517 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.404491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp"] Aug 04 09:52:59.406744 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:52:59.406710 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25bb48f_0b54_422d_9d24_9808f67f8980.slice/crio-d3e43a8f8ef96b6621c86e473b325b6a964fb2017674babd69571798d35f5971 WatchSource:0}: Error finding container d3e43a8f8ef96b6621c86e473b325b6a964fb2017674babd69571798d35f5971: Status 404 returned error can't find the container with id d3e43a8f8ef96b6621c86e473b325b6a964fb2017674babd69571798d35f5971 Aug 04 09:52:59.923700 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.923665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" event={"ID":"c25bb48f-0b54-422d-9d24-9808f67f8980","Type":"ContainerStarted","Data":"e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80"} Aug 04 09:52:59.923909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.923714 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" event={"ID":"c25bb48f-0b54-422d-9d24-9808f67f8980","Type":"ContainerStarted","Data":"d3e43a8f8ef96b6621c86e473b325b6a964fb2017674babd69571798d35f5971"} Aug 04 09:52:59.923909 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.923877 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:52:59.956287 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:52:59.956226 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" podStartSLOduration=1.9562084199999998 podStartE2EDuration="1.95620842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:52:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:52:59.953496921 +0000 UTC m=+2697.085706467" watchObservedRunningTime="2026-08-04 09:52:59.95620842 +0000 UTC m=+2697.088417955" Aug 04 09:53:09.996672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:09.996589 2562 generic.go:356] "Generic (PLEG): container finished" podID="c403c916-d126-40ac-83b6-68a84957e9e6" containerID="79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7" exitCode=0 Aug 04 09:53:09.996672 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:09.996635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" event={"ID":"c403c916-d126-40ac-83b6-68a84957e9e6","Type":"ContainerDied","Data":"79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7"} Aug 04 09:53:09.997107 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:09.996991 2562 scope.go:117] "RemoveContainer" containerID="79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7" Aug 04 09:53:09.998061 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:09.998033 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:53:09.998124 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:09.998101 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:53:09.998168 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:09.998134 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 09:53:10.901897 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:10.901858 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:53:10.932193 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:10.932165 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 09:53:11.006651 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:11.006618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" event={"ID":"c403c916-d126-40ac-83b6-68a84957e9e6","Type":"ContainerStarted","Data":"ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137"} Aug 04 09:53:11.116166 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:11.116133 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:53:11.117406 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:11.117385 2562 patch_prober.go:28] interesting pod/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Aug 04 09:53:11.117513 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:11.117425 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Aug 04 09:53:12.115844 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:12.115804 2562 patch_prober.go:28] interesting pod/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Aug 04 09:53:12.116239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:12.115860 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Aug 04 09:53:13.116474 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:13.116431 2562 patch_prober.go:28] interesting pod/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Aug 04 09:53:13.116844 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:13.116529 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Aug 04 09:53:14.116046 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:14.116005 2562 patch_prober.go:28] interesting pod/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Aug 04 09:53:14.116239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:14.116066 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Aug 04 09:53:15.116096 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:15.116060 2562 patch_prober.go:28] interesting pod/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Aug 04 09:53:15.116579 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:15.116121 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Aug 04 09:53:16.119832 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:16.119801 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:53:16.120310 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:16.120201 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:53:16.120925 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:16.120905 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 09:53:21.965092 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:21.965057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:22.330181 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:22.330080 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:22.681681 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:22.681652 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:22.825398 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:22.825361 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:25.762963 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:25.762923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:25.863443 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:25.863409 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:26.960972 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:26.960932 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:27.158046 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.158013 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:27.617758 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.617721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:27.740191 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.740142 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:27.864293 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.864242 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr"] Aug 04 09:53:27.868130 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.868085 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:27.878786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.878764 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxcsd\" (UniqueName: \"kubernetes.io/projected/f1ab44bf-0de5-4765-8387-44174042b534-kube-api-access-hxcsd\") pod \"payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:27.878874 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.878804 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ab44bf-0de5-4765-8387-44174042b534-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:27.880197 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.880178 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-b0bc81-dockercfg-gjfhs\"" Aug 04 09:53:27.880263 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.880207 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-b0bc81\"" Aug 04 09:53:27.903763 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.903739 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr"] Aug 04 09:53:27.979872 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.979841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxcsd\" (UniqueName: \"kubernetes.io/projected/f1ab44bf-0de5-4765-8387-44174042b534-kube-api-access-hxcsd\") pod \"payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:27.980239 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.979885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ab44bf-0de5-4765-8387-44174042b534-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:27.980580 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.980545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ab44bf-0de5-4765-8387-44174042b534-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:27.992265 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:27.992224 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxcsd\" (UniqueName: \"kubernetes.io/projected/f1ab44bf-0de5-4765-8387-44174042b534-kube-api-access-hxcsd\") pod \"payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:28.178440 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:28.178409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:28.315631 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:28.315601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr"] Aug 04 09:53:28.317208 ip-10-0-136-20 kubenswrapper[2562]: W0804 09:53:28.317176 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ab44bf_0de5_4765_8387_44174042b534.slice/crio-dc2e151d186a640d7a88b1f43b505d42bfefec6ffda7c22a4c192bd5df131ecf WatchSource:0}: Error finding container dc2e151d186a640d7a88b1f43b505d42bfefec6ffda7c22a4c192bd5df131ecf: Status 404 returned error can't find the container with id dc2e151d186a640d7a88b1f43b505d42bfefec6ffda7c22a4c192bd5df131ecf Aug 04 09:53:29.109896 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:29.109860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" event={"ID":"f1ab44bf-0de5-4765-8387-44174042b534","Type":"ContainerStarted","Data":"bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b"} Aug 04 09:53:29.110421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:29.109919 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" event={"ID":"f1ab44bf-0de5-4765-8387-44174042b534","Type":"ContainerStarted","Data":"dc2e151d186a640d7a88b1f43b505d42bfefec6ffda7c22a4c192bd5df131ecf"} Aug 04 09:53:29.110421 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:29.109958 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:29.139803 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:29.139760 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" podStartSLOduration=2.139744526 podStartE2EDuration="2.139744526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:53:29.138547862 +0000 UTC m=+2726.270757395" watchObservedRunningTime="2026-08-04 09:53:29.139744526 +0000 UTC m=+2726.271954057" Aug 04 09:53:40.118084 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:40.118052 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 09:53:50.791880 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:50.791843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:51.668390 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:53:51.668353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:54:02.798880 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:54:02.798847 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:54:16.748008 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:54:16.747974 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:54:37.801745 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:54:37.801698 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:54:37.893636 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:54:37.893603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:03.748265 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:55:03.748219 2562 scope.go:117] "RemoveContainer" containerID="3da7fc66c0885acc93110cca46887937480355f84db0b1b63b506ca55e089ce7" Aug 04 09:56:24.092087 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:56:24.092006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:57:03.777635 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:57:03.777602 2562 scope.go:117] "RemoveContainer" containerID="5fb29c18688b222306fd3ff7c12407df0986895f80e4fdd53ebf5edd44660a95" Aug 04 09:57:37.634786 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:57:37.634750 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:57:38.842450 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:57:38.842384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:58:05.042535 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:58:05.042502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:58:40.802597 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:58:40.802560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:59:03.805980 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:59:03.805904 2562 scope.go:117] "RemoveContainer" containerID="e4354f419323098bb56e63649e3614ce822da86ac6d8232d872bc9f130786877" Aug 04 09:59:13.346192 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:59:13.346156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 09:59:37.070972 ip-10-0-136-20 kubenswrapper[2562]: I0804 09:59:37.070927 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:01.182892 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:00:01.182851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:02.326765 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:00:02.326733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:21.839669 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:00:21.839636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:39.887824 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:00:39.887795 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:01:49.693121 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:49.693086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:01:50.592756 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:50.592721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:01:51.509586 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.509556 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-139849400914992"] Aug 04 10:01:51.513147 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.513128 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:51.515863 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.515844 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-b2tf7\"" Aug 04 10:01:51.523002 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.522980 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400914992"] Aug 04 10:01:51.623512 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.623475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2bxt\" (UniqueName: \"kubernetes.io/projected/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5-kube-api-access-k2bxt\") pod \"test-curl-7693-139849400914992\" (UID: \"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5\") " pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:51.724155 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.724118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2bxt\" (UniqueName: \"kubernetes.io/projected/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5-kube-api-access-k2bxt\") pod \"test-curl-7693-139849400914992\" (UID: \"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5\") " pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:51.733734 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.733706 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2bxt\" (UniqueName: \"kubernetes.io/projected/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5-kube-api-access-k2bxt\") pod \"test-curl-7693-139849400914992\" (UID: \"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5\") " pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:51.822060 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.821978 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:51.955523 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.955493 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400914992"] Aug 04 10:01:51.956906 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:01:51.956869 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf523ea7_c8d7_4c8d_82f8_cefc5e8349c5.slice/crio-cca9c753564e67d3c0c84d6173205763e688317e33d9858b600955a998e0cc74 WatchSource:0}: Error finding container cca9c753564e67d3c0c84d6173205763e688317e33d9858b600955a998e0cc74: Status 404 returned error can't find the container with id cca9c753564e67d3c0c84d6173205763e688317e33d9858b600955a998e0cc74 Aug 04 10:01:51.958203 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:51.958184 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:01:52.945177 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:52.945141 2562 generic.go:356] "Generic (PLEG): container finished" podID="af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" containerID="eefd39255b4166b30ae06d25b26ca976287c474b5e47f920a5c164f1eb637d8c" exitCode=0 Aug 04 10:01:52.945596 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:52.945222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400914992" event={"ID":"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5","Type":"ContainerDied","Data":"eefd39255b4166b30ae06d25b26ca976287c474b5e47f920a5c164f1eb637d8c"} Aug 04 10:01:52.945596 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:52.945271 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400914992" event={"ID":"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5","Type":"ContainerStarted","Data":"cca9c753564e67d3c0c84d6173205763e688317e33d9858b600955a998e0cc74"} Aug 04 10:01:54.087914 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.087888 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:54.112163 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.112138 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7693-139849400914992_af523ea7-c8d7-4c8d-82f8-cefc5e8349c5/test-curl-7693-139849400914992/0.log" Aug 04 10:01:54.138980 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.138953 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-139849400914992"] Aug 04 10:01:54.150344 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.150316 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-139849400914992"] Aug 04 10:01:54.248470 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.248396 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2bxt\" (UniqueName: \"kubernetes.io/projected/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5-kube-api-access-k2bxt\") pod \"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5\" (UID: \"af523ea7-c8d7-4c8d-82f8-cefc5e8349c5\") " Aug 04 10:01:54.250446 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.250414 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5-kube-api-access-k2bxt" (OuterVolumeSpecName: "kube-api-access-k2bxt") pod "af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" (UID: "af523ea7-c8d7-4c8d-82f8-cefc5e8349c5"). InnerVolumeSpecName "kube-api-access-k2bxt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:01:54.278167 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.278137 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-139849400952368"] Aug 04 10:01:54.278561 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.278547 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" containerName="test-curl-7693-139849400914992" Aug 04 10:01:54.278621 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.278563 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" containerName="test-curl-7693-139849400914992" Aug 04 10:01:54.278661 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.278622 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" containerName="test-curl-7693-139849400914992" Aug 04 10:01:54.281631 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.281617 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:54.301351 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.301327 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400952368"] Aug 04 10:01:54.349246 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.349219 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2bxt\" (UniqueName: \"kubernetes.io/projected/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5-kube-api-access-k2bxt\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:01:54.450116 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.450085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l8w7\" (UniqueName: \"kubernetes.io/projected/616f5aa1-3af7-4529-bc81-7c2a79f052cc-kube-api-access-6l8w7\") pod \"test-curl-7693-139849400952368\" (UID: \"616f5aa1-3af7-4529-bc81-7c2a79f052cc\") " pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:54.551051 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.550963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l8w7\" (UniqueName: \"kubernetes.io/projected/616f5aa1-3af7-4529-bc81-7c2a79f052cc-kube-api-access-6l8w7\") pod \"test-curl-7693-139849400952368\" (UID: \"616f5aa1-3af7-4529-bc81-7c2a79f052cc\") " pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:54.561803 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.561775 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l8w7\" (UniqueName: \"kubernetes.io/projected/616f5aa1-3af7-4529-bc81-7c2a79f052cc-kube-api-access-6l8w7\") pod \"test-curl-7693-139849400952368\" (UID: \"616f5aa1-3af7-4529-bc81-7c2a79f052cc\") " pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:54.590645 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.590619 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:54.747614 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.747585 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400952368"] Aug 04 10:01:54.749312 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:01:54.749283 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod616f5aa1_3af7_4529_bc81_7c2a79f052cc.slice/crio-ff42cb825c8c6207cb5c4b75ac918c7e1a8aaaaa0907da1707259b27a0a401eb WatchSource:0}: Error finding container ff42cb825c8c6207cb5c4b75ac918c7e1a8aaaaa0907da1707259b27a0a401eb: Status 404 returned error can't find the container with id ff42cb825c8c6207cb5c4b75ac918c7e1a8aaaaa0907da1707259b27a0a401eb Aug 04 10:01:54.957733 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.957697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400952368" event={"ID":"616f5aa1-3af7-4529-bc81-7c2a79f052cc","Type":"ContainerStarted","Data":"ff42cb825c8c6207cb5c4b75ac918c7e1a8aaaaa0907da1707259b27a0a401eb"} Aug 04 10:01:54.958945 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.958921 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:54.959060 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:54.958954 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cca9c753564e67d3c0c84d6173205763e688317e33d9858b600955a998e0cc74" Aug 04 10:01:54.970025 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:01:54.970001 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-139849400914992\" is forbidden: User \"system:node:ip-10-0-136-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-20.ec2.internal' and this object" podUID="af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" pod="opendatahub/test-curl-7693-139849400914992" Aug 04 10:01:55.416598 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:55.416567 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af523ea7-c8d7-4c8d-82f8-cefc5e8349c5" path="/var/lib/kubelet/pods/af523ea7-c8d7-4c8d-82f8-cefc5e8349c5/volumes" Aug 04 10:01:55.965427 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:55.965395 2562 generic.go:356] "Generic (PLEG): container finished" podID="616f5aa1-3af7-4529-bc81-7c2a79f052cc" containerID="4959fa5ceaf166ff324281b1a7b0f77674c6ac18e844fd94202e7914dfab7eda" exitCode=0 Aug 04 10:01:55.965624 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:55.965473 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400952368" event={"ID":"616f5aa1-3af7-4529-bc81-7c2a79f052cc","Type":"ContainerDied","Data":"4959fa5ceaf166ff324281b1a7b0f77674c6ac18e844fd94202e7914dfab7eda"} Aug 04 10:01:57.102738 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.102713 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:57.126542 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.126521 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7693-139849400952368_616f5aa1-3af7-4529-bc81-7c2a79f052cc/test-curl-7693-139849400952368/0.log" Aug 04 10:01:57.153891 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.153865 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-139849400952368"] Aug 04 10:01:57.160831 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.160808 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-139849400952368"] Aug 04 10:01:57.277371 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.277296 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l8w7\" (UniqueName: \"kubernetes.io/projected/616f5aa1-3af7-4529-bc81-7c2a79f052cc-kube-api-access-6l8w7\") pod \"616f5aa1-3af7-4529-bc81-7c2a79f052cc\" (UID: \"616f5aa1-3af7-4529-bc81-7c2a79f052cc\") " Aug 04 10:01:57.279360 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.279331 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/616f5aa1-3af7-4529-bc81-7c2a79f052cc-kube-api-access-6l8w7" (OuterVolumeSpecName: "kube-api-access-6l8w7") pod "616f5aa1-3af7-4529-bc81-7c2a79f052cc" (UID: "616f5aa1-3af7-4529-bc81-7c2a79f052cc"). InnerVolumeSpecName "kube-api-access-6l8w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:01:57.333969 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.333937 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:01:57.334350 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.334335 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="616f5aa1-3af7-4529-bc81-7c2a79f052cc" containerName="test-curl-7693-139849400952368" Aug 04 10:01:57.334407 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.334352 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="616f5aa1-3af7-4529-bc81-7c2a79f052cc" containerName="test-curl-7693-139849400952368" Aug 04 10:01:57.334448 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.334426 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="616f5aa1-3af7-4529-bc81-7c2a79f052cc" containerName="test-curl-7693-139849400952368" Aug 04 10:01:57.342799 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.342770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:01:57.346478 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.346453 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:01:57.378174 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.378150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sns47\" (UniqueName: \"kubernetes.io/projected/9c5643ca-52db-4986-a1d5-776172999f17-kube-api-access-sns47\") pod \"test-curl-7693-139849400778016\" (UID: \"9c5643ca-52db-4986-a1d5-776172999f17\") " pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:01:57.378359 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.378340 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l8w7\" (UniqueName: \"kubernetes.io/projected/616f5aa1-3af7-4529-bc81-7c2a79f052cc-kube-api-access-6l8w7\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:01:57.416274 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.416233 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="616f5aa1-3af7-4529-bc81-7c2a79f052cc" path="/var/lib/kubelet/pods/616f5aa1-3af7-4529-bc81-7c2a79f052cc/volumes" Aug 04 10:01:57.478755 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.478730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sns47\" (UniqueName: \"kubernetes.io/projected/9c5643ca-52db-4986-a1d5-776172999f17-kube-api-access-sns47\") pod \"test-curl-7693-139849400778016\" (UID: \"9c5643ca-52db-4986-a1d5-776172999f17\") " pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:01:57.488970 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.488943 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sns47\" (UniqueName: \"kubernetes.io/projected/9c5643ca-52db-4986-a1d5-776172999f17-kube-api-access-sns47\") pod \"test-curl-7693-139849400778016\" (UID: \"9c5643ca-52db-4986-a1d5-776172999f17\") " pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:01:57.653814 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.653791 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:01:57.787425 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.787396 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:01:57.789011 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:01:57.788976 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c5643ca_52db_4986_a1d5_776172999f17.slice/crio-a2603e50832a325ea9cdf9bba5b5e3c7923d72c2ebd03d4fdee54ea2e0f2ac7b WatchSource:0}: Error finding container a2603e50832a325ea9cdf9bba5b5e3c7923d72c2ebd03d4fdee54ea2e0f2ac7b: Status 404 returned error can't find the container with id a2603e50832a325ea9cdf9bba5b5e3c7923d72c2ebd03d4fdee54ea2e0f2ac7b Aug 04 10:01:57.977516 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.977423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400778016" event={"ID":"9c5643ca-52db-4986-a1d5-776172999f17","Type":"ContainerStarted","Data":"a2603e50832a325ea9cdf9bba5b5e3c7923d72c2ebd03d4fdee54ea2e0f2ac7b"} Aug 04 10:01:57.978638 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.978614 2562 scope.go:117] "RemoveContainer" containerID="4959fa5ceaf166ff324281b1a7b0f77674c6ac18e844fd94202e7914dfab7eda" Aug 04 10:01:57.978638 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:57.978626 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400952368" Aug 04 10:01:58.986338 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:58.986303 2562 generic.go:356] "Generic (PLEG): container finished" podID="9c5643ca-52db-4986-a1d5-776172999f17" containerID="68010996fc0b3d97d61f9336fac2e1acee679e34c6e815232fbd9ef5ad3e7968" exitCode=0 Aug 04 10:01:58.986717 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:01:58.986377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400778016" event={"ID":"9c5643ca-52db-4986-a1d5-776172999f17","Type":"ContainerDied","Data":"68010996fc0b3d97d61f9336fac2e1acee679e34c6e815232fbd9ef5ad3e7968"} Aug 04 10:02:00.125964 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.125940 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:00.151876 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.151853 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7693-139849400778016_9c5643ca-52db-4986-a1d5-776172999f17/test-curl-7693-139849400778016/0.log" Aug 04 10:02:00.180607 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.180577 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:00.182811 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.182790 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:00.200123 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.200101 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sns47\" (UniqueName: \"kubernetes.io/projected/9c5643ca-52db-4986-a1d5-776172999f17-kube-api-access-sns47\") pod \"9c5643ca-52db-4986-a1d5-776172999f17\" (UID: \"9c5643ca-52db-4986-a1d5-776172999f17\") " Aug 04 10:02:00.202130 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.202095 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5643ca-52db-4986-a1d5-776172999f17-kube-api-access-sns47" (OuterVolumeSpecName: "kube-api-access-sns47") pod "9c5643ca-52db-4986-a1d5-776172999f17" (UID: "9c5643ca-52db-4986-a1d5-776172999f17"). InnerVolumeSpecName "kube-api-access-sns47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:02:00.300886 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.300804 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sns47\" (UniqueName: \"kubernetes.io/projected/9c5643ca-52db-4986-a1d5-776172999f17-kube-api-access-sns47\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:02:00.320108 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.320075 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:00.320591 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.320573 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c5643ca-52db-4986-a1d5-776172999f17" containerName="test-curl-7693-139849400778016" Aug 04 10:02:00.320692 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.320593 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c5643ca-52db-4986-a1d5-776172999f17" containerName="test-curl-7693-139849400778016" Aug 04 10:02:00.320692 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.320687 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c5643ca-52db-4986-a1d5-776172999f17" containerName="test-curl-7693-139849400778016" Aug 04 10:02:00.323840 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.323820 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:00.333504 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.333478 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:00.401946 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.401918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69rjq\" (UniqueName: \"kubernetes.io/projected/36f87e26-53d5-4f62-b1e8-89b37ef721c7-kube-api-access-69rjq\") pod \"test-curl-7693-139849400778016\" (UID: \"36f87e26-53d5-4f62-b1e8-89b37ef721c7\") " pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:00.502904 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.502870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69rjq\" (UniqueName: \"kubernetes.io/projected/36f87e26-53d5-4f62-b1e8-89b37ef721c7-kube-api-access-69rjq\") pod \"test-curl-7693-139849400778016\" (UID: \"36f87e26-53d5-4f62-b1e8-89b37ef721c7\") " pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:00.512902 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.512875 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69rjq\" (UniqueName: \"kubernetes.io/projected/36f87e26-53d5-4f62-b1e8-89b37ef721c7-kube-api-access-69rjq\") pod \"test-curl-7693-139849400778016\" (UID: \"36f87e26-53d5-4f62-b1e8-89b37ef721c7\") " pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:00.634120 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.634042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:00.779135 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.779109 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:00.779262 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:00.779118 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36f87e26_53d5_4f62_b1e8_89b37ef721c7.slice/crio-49176cbee651c9032e8d486c132384ab1e378a83a203c7312cc17248a23054cf WatchSource:0}: Error finding container 49176cbee651c9032e8d486c132384ab1e378a83a203c7312cc17248a23054cf: Status 404 returned error can't find the container with id 49176cbee651c9032e8d486c132384ab1e378a83a203c7312cc17248a23054cf Aug 04 10:02:00.999775 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:00.999731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400778016" event={"ID":"36f87e26-53d5-4f62-b1e8-89b37ef721c7","Type":"ContainerStarted","Data":"49176cbee651c9032e8d486c132384ab1e378a83a203c7312cc17248a23054cf"} Aug 04 10:02:01.001060 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:01.001034 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2603e50832a325ea9cdf9bba5b5e3c7923d72c2ebd03d4fdee54ea2e0f2ac7b" Aug 04 10:02:01.001060 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:01.001034 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:01.012295 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:01.012239 2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7693-139849400778016" oldPodUID="9c5643ca-52db-4986-a1d5-776172999f17" podUID="36f87e26-53d5-4f62-b1e8-89b37ef721c7" Aug 04 10:02:01.417661 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:01.417620 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c5643ca-52db-4986-a1d5-776172999f17" path="/var/lib/kubelet/pods/9c5643ca-52db-4986-a1d5-776172999f17/volumes" Aug 04 10:02:02.007445 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:02.007407 2562 generic.go:356] "Generic (PLEG): container finished" podID="36f87e26-53d5-4f62-b1e8-89b37ef721c7" containerID="f9ce636203b19c86e75ad3b06089dc9a294038092cc3475ed506c065048dba21" exitCode=0 Aug 04 10:02:02.007617 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:02.007458 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400778016" event={"ID":"36f87e26-53d5-4f62-b1e8-89b37ef721c7","Type":"ContainerDied","Data":"f9ce636203b19c86e75ad3b06089dc9a294038092cc3475ed506c065048dba21"} Aug 04 10:02:03.154829 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.154800 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:03.178108 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.178086 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7693-139849400778016_36f87e26-53d5-4f62-b1e8-89b37ef721c7/test-curl-7693-139849400778016/0.log" Aug 04 10:02:03.207535 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.207508 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:03.209892 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.209870 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-139849400778016"] Aug 04 10:02:03.222588 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.222563 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69rjq\" (UniqueName: \"kubernetes.io/projected/36f87e26-53d5-4f62-b1e8-89b37ef721c7-kube-api-access-69rjq\") pod \"36f87e26-53d5-4f62-b1e8-89b37ef721c7\" (UID: \"36f87e26-53d5-4f62-b1e8-89b37ef721c7\") " Aug 04 10:02:03.224503 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.224482 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f87e26-53d5-4f62-b1e8-89b37ef721c7-kube-api-access-69rjq" (OuterVolumeSpecName: "kube-api-access-69rjq") pod "36f87e26-53d5-4f62-b1e8-89b37ef721c7" (UID: "36f87e26-53d5-4f62-b1e8-89b37ef721c7"). InnerVolumeSpecName "kube-api-access-69rjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:02:03.323285 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.323193 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69rjq\" (UniqueName: \"kubernetes.io/projected/36f87e26-53d5-4f62-b1e8-89b37ef721c7-kube-api-access-69rjq\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:02:03.339801 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.339772 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:03.340175 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.340161 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f87e26-53d5-4f62-b1e8-89b37ef721c7" containerName="test-curl-7693-139849400778016" Aug 04 10:02:03.340229 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.340176 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f87e26-53d5-4f62-b1e8-89b37ef721c7" containerName="test-curl-7693-139849400778016" Aug 04 10:02:03.340299 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.340245 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f87e26-53d5-4f62-b1e8-89b37ef721c7" containerName="test-curl-7693-139849400778016" Aug 04 10:02:03.346466 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.346450 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:03.353493 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.353472 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:03.416830 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.416796 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36f87e26-53d5-4f62-b1e8-89b37ef721c7" path="/var/lib/kubelet/pods/36f87e26-53d5-4f62-b1e8-89b37ef721c7/volumes" Aug 04 10:02:03.423939 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.423908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qcd5\" (UniqueName: \"kubernetes.io/projected/eec53e97-77ae-49f7-b8e6-93947d93f543-kube-api-access-5qcd5\") pod \"test-curl-7693-139849400877392\" (UID: \"eec53e97-77ae-49f7-b8e6-93947d93f543\") " pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:03.524832 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.524802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qcd5\" (UniqueName: \"kubernetes.io/projected/eec53e97-77ae-49f7-b8e6-93947d93f543-kube-api-access-5qcd5\") pod \"test-curl-7693-139849400877392\" (UID: \"eec53e97-77ae-49f7-b8e6-93947d93f543\") " pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:03.533637 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.533612 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qcd5\" (UniqueName: \"kubernetes.io/projected/eec53e97-77ae-49f7-b8e6-93947d93f543-kube-api-access-5qcd5\") pod \"test-curl-7693-139849400877392\" (UID: \"eec53e97-77ae-49f7-b8e6-93947d93f543\") " pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:03.656603 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.656575 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:03.789322 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:03.789290 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:03.790134 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:03.790101 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeec53e97_77ae_49f7_b8e6_93947d93f543.slice/crio-7d81fd378ac9fdbd8c7444303daf7f87e22f8447cd2b6f7879affd3c4da8345f WatchSource:0}: Error finding container 7d81fd378ac9fdbd8c7444303daf7f87e22f8447cd2b6f7879affd3c4da8345f: Status 404 returned error can't find the container with id 7d81fd378ac9fdbd8c7444303daf7f87e22f8447cd2b6f7879affd3c4da8345f Aug 04 10:02:04.021087 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:04.020999 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400778016" Aug 04 10:02:04.021087 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:04.021043 2562 scope.go:117] "RemoveContainer" containerID="f9ce636203b19c86e75ad3b06089dc9a294038092cc3475ed506c065048dba21" Aug 04 10:02:04.022385 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:04.022365 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400877392" event={"ID":"eec53e97-77ae-49f7-b8e6-93947d93f543","Type":"ContainerStarted","Data":"7d81fd378ac9fdbd8c7444303daf7f87e22f8447cd2b6f7879affd3c4da8345f"} Aug 04 10:02:05.030391 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:05.030358 2562 generic.go:356] "Generic (PLEG): container finished" podID="eec53e97-77ae-49f7-b8e6-93947d93f543" containerID="eabf9d3481965592839ce185d2007923bfc14d654cf58042424d6739a8c1bbc1" exitCode=0 Aug 04 10:02:05.030794 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:05.030439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400877392" event={"ID":"eec53e97-77ae-49f7-b8e6-93947d93f543","Type":"ContainerDied","Data":"eabf9d3481965592839ce185d2007923bfc14d654cf58042424d6739a8c1bbc1"} Aug 04 10:02:06.172077 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.172051 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:06.194237 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.194202 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7693-139849400877392_eec53e97-77ae-49f7-b8e6-93947d93f543/test-curl-7693-139849400877392/0.log" Aug 04 10:02:06.221861 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.221824 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:06.234158 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.234126 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:06.254175 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.254138 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qcd5\" (UniqueName: \"kubernetes.io/projected/eec53e97-77ae-49f7-b8e6-93947d93f543-kube-api-access-5qcd5\") pod \"eec53e97-77ae-49f7-b8e6-93947d93f543\" (UID: \"eec53e97-77ae-49f7-b8e6-93947d93f543\") " Aug 04 10:02:06.256154 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.256132 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eec53e97-77ae-49f7-b8e6-93947d93f543-kube-api-access-5qcd5" (OuterVolumeSpecName: "kube-api-access-5qcd5") pod "eec53e97-77ae-49f7-b8e6-93947d93f543" (UID: "eec53e97-77ae-49f7-b8e6-93947d93f543"). InnerVolumeSpecName "kube-api-access-5qcd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:02:06.355890 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.355807 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qcd5\" (UniqueName: \"kubernetes.io/projected/eec53e97-77ae-49f7-b8e6-93947d93f543-kube-api-access-5qcd5\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:02:06.359026 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.358990 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:06.359449 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.359435 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eec53e97-77ae-49f7-b8e6-93947d93f543" containerName="test-curl-7693-139849400877392" Aug 04 10:02:06.359496 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.359451 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec53e97-77ae-49f7-b8e6-93947d93f543" containerName="test-curl-7693-139849400877392" Aug 04 10:02:06.359533 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.359510 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eec53e97-77ae-49f7-b8e6-93947d93f543" containerName="test-curl-7693-139849400877392" Aug 04 10:02:06.362609 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.362595 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:06.374903 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.374872 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:06.456858 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.456825 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zrkc\" (UniqueName: \"kubernetes.io/projected/3c1c5859-32a2-4485-a0d8-625597f7aa23-kube-api-access-2zrkc\") pod \"test-curl-7693-139849400877392\" (UID: \"3c1c5859-32a2-4485-a0d8-625597f7aa23\") " pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:06.557859 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.557821 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrkc\" (UniqueName: \"kubernetes.io/projected/3c1c5859-32a2-4485-a0d8-625597f7aa23-kube-api-access-2zrkc\") pod \"test-curl-7693-139849400877392\" (UID: \"3c1c5859-32a2-4485-a0d8-625597f7aa23\") " pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:06.572461 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.572438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrkc\" (UniqueName: \"kubernetes.io/projected/3c1c5859-32a2-4485-a0d8-625597f7aa23-kube-api-access-2zrkc\") pod \"test-curl-7693-139849400877392\" (UID: \"3c1c5859-32a2-4485-a0d8-625597f7aa23\") " pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:06.671390 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.671359 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:06.804613 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:06.804590 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:06.805933 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:06.805902 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c1c5859_32a2_4485_a0d8_625597f7aa23.slice/crio-291c56394bc7403e50e35bbbbf52e25b35c98554441c4f2ca4b46d0a2e4438f3 WatchSource:0}: Error finding container 291c56394bc7403e50e35bbbbf52e25b35c98554441c4f2ca4b46d0a2e4438f3: Status 404 returned error can't find the container with id 291c56394bc7403e50e35bbbbf52e25b35c98554441c4f2ca4b46d0a2e4438f3 Aug 04 10:02:07.044022 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:07.043926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400877392" event={"ID":"3c1c5859-32a2-4485-a0d8-625597f7aa23","Type":"ContainerStarted","Data":"291c56394bc7403e50e35bbbbf52e25b35c98554441c4f2ca4b46d0a2e4438f3"} Aug 04 10:02:07.045225 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:07.045201 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:07.045350 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:07.045209 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d81fd378ac9fdbd8c7444303daf7f87e22f8447cd2b6f7879affd3c4da8345f" Aug 04 10:02:07.057062 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:07.057037 2562 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7693-139849400877392" oldPodUID="eec53e97-77ae-49f7-b8e6-93947d93f543" podUID="3c1c5859-32a2-4485-a0d8-625597f7aa23" Aug 04 10:02:07.416000 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:07.415969 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eec53e97-77ae-49f7-b8e6-93947d93f543" path="/var/lib/kubelet/pods/eec53e97-77ae-49f7-b8e6-93947d93f543/volumes" Aug 04 10:02:08.051792 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:08.051757 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c1c5859-32a2-4485-a0d8-625597f7aa23" containerID="6fb93dd96e021032db3c499243df5646c320e36937261d6f6eb5cc521ff7d6c3" exitCode=0 Aug 04 10:02:08.051973 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:08.051819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-139849400877392" event={"ID":"3c1c5859-32a2-4485-a0d8-625597f7aa23","Type":"ContainerDied","Data":"6fb93dd96e021032db3c499243df5646c320e36937261d6f6eb5cc521ff7d6c3"} Aug 04 10:02:09.189935 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.189911 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:09.217690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.217664 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7693-139849400877392_3c1c5859-32a2-4485-a0d8-625597f7aa23/test-curl-7693-139849400877392/0.log" Aug 04 10:02:09.244071 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.244039 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:09.247951 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.247926 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-139849400877392"] Aug 04 10:02:09.283787 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.283758 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zrkc\" (UniqueName: \"kubernetes.io/projected/3c1c5859-32a2-4485-a0d8-625597f7aa23-kube-api-access-2zrkc\") pod \"3c1c5859-32a2-4485-a0d8-625597f7aa23\" (UID: \"3c1c5859-32a2-4485-a0d8-625597f7aa23\") " Aug 04 10:02:09.285941 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.285917 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1c5859-32a2-4485-a0d8-625597f7aa23-kube-api-access-2zrkc" (OuterVolumeSpecName: "kube-api-access-2zrkc") pod "3c1c5859-32a2-4485-a0d8-625597f7aa23" (UID: "3c1c5859-32a2-4485-a0d8-625597f7aa23"). InnerVolumeSpecName "kube-api-access-2zrkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:02:09.385293 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.385184 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zrkc\" (UniqueName: \"kubernetes.io/projected/3c1c5859-32a2-4485-a0d8-625597f7aa23-kube-api-access-2zrkc\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:02:09.416707 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.416675 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c1c5859-32a2-4485-a0d8-625597f7aa23" path="/var/lib/kubelet/pods/3c1c5859-32a2-4485-a0d8-625597f7aa23/volumes" Aug 04 10:02:09.780788 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:09.780755 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:10.064814 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.064742 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-139849400877392" Aug 04 10:02:10.064814 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.064761 2562 scope.go:117] "RemoveContainer" containerID="6fb93dd96e021032db3c499243df5646c320e36937261d6f6eb5cc521ff7d6c3" Aug 04 10:02:10.134528 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.134501 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8"] Aug 04 10:02:10.134908 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.134894 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c1c5859-32a2-4485-a0d8-625597f7aa23" containerName="test-curl-7693-139849400877392" Aug 04 10:02:10.134954 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.134909 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c1c5859-32a2-4485-a0d8-625597f7aa23" containerName="test-curl-7693-139849400877392" Aug 04 10:02:10.134987 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.134969 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c1c5859-32a2-4485-a0d8-625597f7aa23" containerName="test-curl-7693-139849400877392" Aug 04 10:02:10.139427 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.139410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.143645 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.143629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-fc7b93-openshift-default-dockercfg-kvv8j\"" Aug 04 10:02:10.155193 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.155170 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8"] Aug 04 10:02:10.192237 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-certs\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-data\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-credential-socket\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-envoy\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2mkv\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-kube-api-access-s2mkv\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istiod-ca-cert\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-podinfo\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192496 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-token\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.192582 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.192533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-socket\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.193466 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.193449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:10.293335 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-data\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.293519 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-credential-socket\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.293519 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-envoy\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.293519 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2mkv\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-kube-api-access-s2mkv\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.293519 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293417 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istiod-ca-cert\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.293519 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-podinfo\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294006 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-token\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294006 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-socket\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294006 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-certs\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294006 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293726 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-credential-socket\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294006 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.293797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-data\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294218 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.294043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-socket\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294218 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.294047 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-certs\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.294218 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.294074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istiod-ca-cert\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.295718 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.295698 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-envoy\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.296053 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.296031 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-podinfo\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.303707 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.303684 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-token\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.304117 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.304097 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2mkv\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-kube-api-access-s2mkv\") pod \"e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.449480 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.449447 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:10.590664 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.590636 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8"] Aug 04 10:02:10.591699 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:10.591671 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe82bf0_b6c7_4a1d_9cd1_304eed0a16d6.slice/crio-e8b72bb6bdd8992b871cdc81c4d9a77d09d12843f1daa84a713afe9ad7203f00 WatchSource:0}: Error finding container e8b72bb6bdd8992b871cdc81c4d9a77d09d12843f1daa84a713afe9ad7203f00: Status 404 returned error can't find the container with id e8b72bb6bdd8992b871cdc81c4d9a77d09d12843f1daa84a713afe9ad7203f00 Aug 04 10:02:10.594460 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.594428 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:02:10.594578 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.594490 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:02:10.594578 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.594529 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:02:10.595303 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:10.595275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:11.072725 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:11.072691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" event={"ID":"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6","Type":"ContainerStarted","Data":"66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa"} Aug 04 10:02:11.072897 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:11.072784 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" event={"ID":"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6","Type":"ContainerStarted","Data":"e8b72bb6bdd8992b871cdc81c4d9a77d09d12843f1daa84a713afe9ad7203f00"} Aug 04 10:02:11.100294 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:11.100229 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podStartSLOduration=1.100213815 podStartE2EDuration="1.100213815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:02:11.09936766 +0000 UTC m=+3248.231577192" watchObservedRunningTime="2026-08-04 10:02:11.100213815 +0000 UTC m=+3248.232423346" Aug 04 10:02:11.450096 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:11.450066 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:11.455658 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:11.455637 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:12.079887 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:12.079855 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:12.080830 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:12.080812 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:13.117542 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:13.117488 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:13.212042 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:13.212003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:14.246991 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:14.246959 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:14.979939 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:14.979906 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2"] Aug 04 10:02:14.983696 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:14.983680 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:14.986296 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:14.986270 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fc7b93\"" Aug 04 10:02:14.986457 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:14.986441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fc7b93-dockercfg-mtgdz\"" Aug 04 10:02:15.001097 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.001074 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2"] Aug 04 10:02:15.036692 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.036668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a8972da-b2a7-4df0-8873-4f9b68b68026-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.036790 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.036709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf242\" (UniqueName: \"kubernetes.io/projected/7a8972da-b2a7-4df0-8873-4f9b68b68026-kube-api-access-pf242\") pod \"payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.042701 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.042680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:15.137641 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.137609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a8972da-b2a7-4df0-8873-4f9b68b68026-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.137816 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.137655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf242\" (UniqueName: \"kubernetes.io/projected/7a8972da-b2a7-4df0-8873-4f9b68b68026-kube-api-access-pf242\") pod \"payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.138325 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.138306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a8972da-b2a7-4df0-8873-4f9b68b68026-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.149915 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.149889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf242\" (UniqueName: \"kubernetes.io/projected/7a8972da-b2a7-4df0-8873-4f9b68b68026-kube-api-access-pf242\") pod \"payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.186546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.186495 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:15.294129 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.294051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:15.430809 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:15.430783 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2"] Aug 04 10:02:15.431701 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:15.431678 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8972da_b2a7_4df0_8873_4f9b68b68026.slice/crio-3c81afc7131a74b4553ed5d91897e8c3da0ab3adf9ce773749f82a097f775c2f WatchSource:0}: Error finding container 3c81afc7131a74b4553ed5d91897e8c3da0ab3adf9ce773749f82a097f775c2f: Status 404 returned error can't find the container with id 3c81afc7131a74b4553ed5d91897e8c3da0ab3adf9ce773749f82a097f775c2f Aug 04 10:02:16.104843 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:16.104805 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" event={"ID":"7a8972da-b2a7-4df0-8873-4f9b68b68026","Type":"ContainerStarted","Data":"8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293"} Aug 04 10:02:16.105018 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:16.104886 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:16.105018 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:16.104909 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" event={"ID":"7a8972da-b2a7-4df0-8873-4f9b68b68026","Type":"ContainerStarted","Data":"3c81afc7131a74b4553ed5d91897e8c3da0ab3adf9ce773749f82a097f775c2f"} Aug 04 10:02:16.125656 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:16.125550 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" podStartSLOduration=2.125532898 podStartE2EDuration="2.125532898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:02:16.124626393 +0000 UTC m=+3253.256835927" watchObservedRunningTime="2026-08-04 10:02:16.125532898 +0000 UTC m=+3253.257742431" Aug 04 10:02:26.166927 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:26.166894 2562 generic.go:356] "Generic (PLEG): container finished" podID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerID="66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa" exitCode=0 Aug 04 10:02:26.167379 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:26.166967 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" event={"ID":"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6","Type":"ContainerDied","Data":"66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa"} Aug 04 10:02:26.167436 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:26.167422 2562 scope.go:117] "RemoveContainer" containerID="66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa" Aug 04 10:02:26.168489 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:26.168461 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:02:26.168608 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:26.168527 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:02:26.168608 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:26.168559 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:02:27.079819 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:27.079769 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:27.113756 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:27.113720 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:02:27.180675 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:27.180646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" event={"ID":"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6","Type":"ContainerStarted","Data":"76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb"} Aug 04 10:02:27.449838 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:27.449801 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:27.450738 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:27.450705 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body= Aug 04 10:02:27.450865 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:27.450760 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" Aug 04 10:02:28.450642 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:28.450598 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body= Aug 04 10:02:28.451136 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:28.450672 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" Aug 04 10:02:29.450405 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:29.450362 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body= Aug 04 10:02:29.450616 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:29.450433 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" Aug 04 10:02:30.450830 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:30.450783 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body= Aug 04 10:02:30.451350 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:30.450851 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" Aug 04 10:02:31.450871 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:31.450826 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body= Aug 04 10:02:31.451384 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:31.450898 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" Aug 04 10:02:32.454588 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:32.454559 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:32.454951 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:32.454875 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:32.455581 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:32.455557 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:02:39.010096 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:39.010061 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:40.544013 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:40.543971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:42.415643 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:42.415601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:42.509147 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:42.509115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:43.722271 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:43.722227 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:44.063614 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.063512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:44.271125 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.271090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:44.372506 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.372423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:02:44.395360 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.395330 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw"] Aug 04 10:02:44.398974 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.398958 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.402571 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.402545 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-90d908\"" Aug 04 10:02:44.418461 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.418437 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw"] Aug 04 10:02:44.438263 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.438225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ad9fb0-32ad-4209-a565-3d72ed4964d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.438457 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.438433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzhs9\" (UniqueName: \"kubernetes.io/projected/39ad9fb0-32ad-4209-a565-3d72ed4964d5-kube-api-access-rzhs9\") pod \"payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.473203 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.473157 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9"] Aug 04 10:02:44.476458 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.476444 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.478941 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.478924 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-90d908-dockercfg-b42z8\"" Aug 04 10:02:44.487566 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.487545 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9"] Aug 04 10:02:44.539242 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.539188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ad9fb0-32ad-4209-a565-3d72ed4964d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.539242 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.539266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pftbh\" (UniqueName: \"kubernetes.io/projected/36b78755-0836-46d8-9bae-8ffb47f2d406-kube-api-access-pftbh\") pod \"payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.539620 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.539351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b78755-0836-46d8-9bae-8ffb47f2d406-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.539620 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.539384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzhs9\" (UniqueName: \"kubernetes.io/projected/39ad9fb0-32ad-4209-a565-3d72ed4964d5-kube-api-access-rzhs9\") pod \"payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.540204 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.540177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ad9fb0-32ad-4209-a565-3d72ed4964d5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.552126 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.552099 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzhs9\" (UniqueName: \"kubernetes.io/projected/39ad9fb0-32ad-4209-a565-3d72ed4964d5-kube-api-access-rzhs9\") pod \"payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.640179 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.640098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pftbh\" (UniqueName: \"kubernetes.io/projected/36b78755-0836-46d8-9bae-8ffb47f2d406-kube-api-access-pftbh\") pod \"payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.640338 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.640185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b78755-0836-46d8-9bae-8ffb47f2d406-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.640906 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.640885 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b78755-0836-46d8-9bae-8ffb47f2d406-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.649402 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.649376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pftbh\" (UniqueName: \"kubernetes.io/projected/36b78755-0836-46d8-9bae-8ffb47f2d406-kube-api-access-pftbh\") pod \"payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.709827 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.709801 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:44.787100 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.787061 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:44.869086 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:44.869061 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw"] Aug 04 10:02:44.869568 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:44.869539 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ad9fb0_32ad_4209_a565_3d72ed4964d5.slice/crio-125bd3d3c2319b61892be63070fd55f49e0153c6b0d801067c90fb4eb88416d5 WatchSource:0}: Error finding container 125bd3d3c2319b61892be63070fd55f49e0153c6b0d801067c90fb4eb88416d5: Status 404 returned error can't find the container with id 125bd3d3c2319b61892be63070fd55f49e0153c6b0d801067c90fb4eb88416d5 Aug 04 10:02:45.166613 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.166588 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9"] Aug 04 10:02:45.168298 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:02:45.168270 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b78755_0836_46d8_9bae_8ffb47f2d406.slice/crio-e0080d5704f4c00e942b4b00555a2834fefe915f5674ccdbcece5bb18f32d959 WatchSource:0}: Error finding container e0080d5704f4c00e942b4b00555a2834fefe915f5674ccdbcece5bb18f32d959: Status 404 returned error can't find the container with id e0080d5704f4c00e942b4b00555a2834fefe915f5674ccdbcece5bb18f32d959 Aug 04 10:02:45.291242 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.291199 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" event={"ID":"39ad9fb0-32ad-4209-a565-3d72ed4964d5","Type":"ContainerStarted","Data":"19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da"} Aug 04 10:02:45.291429 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.291281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" event={"ID":"39ad9fb0-32ad-4209-a565-3d72ed4964d5","Type":"ContainerStarted","Data":"125bd3d3c2319b61892be63070fd55f49e0153c6b0d801067c90fb4eb88416d5"} Aug 04 10:02:45.291485 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.291459 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:45.293185 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.293161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" event={"ID":"36b78755-0836-46d8-9bae-8ffb47f2d406","Type":"ContainerStarted","Data":"ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9"} Aug 04 10:02:45.293345 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.293206 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" event={"ID":"36b78755-0836-46d8-9bae-8ffb47f2d406","Type":"ContainerStarted","Data":"e0080d5704f4c00e942b4b00555a2834fefe915f5674ccdbcece5bb18f32d959"} Aug 04 10:02:45.293345 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.293228 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:02:45.309321 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.309279 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" podStartSLOduration=1.3092483480000001 podStartE2EDuration="1.309248348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:02:45.308458181 +0000 UTC m=+3282.440667719" watchObservedRunningTime="2026-08-04 10:02:45.309248348 +0000 UTC m=+3282.441473819" Aug 04 10:02:45.332992 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:45.332949 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" podStartSLOduration=1.332937021 podStartE2EDuration="1.332937021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:02:45.331688466 +0000 UTC m=+3282.463897998" watchObservedRunningTime="2026-08-04 10:02:45.332937021 +0000 UTC m=+3282.465146552" Aug 04 10:02:56.301675 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:56.301626 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:02:56.302147 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:02:56.301731 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:03:03.552396 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:03.552363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:03:06.288404 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:06.288366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:03:06.461217 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:06.461185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:03:07.507652 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:07.507615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:03:14.582275 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:14.582218 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw_39ad9fb0-32ad-4209-a565-3d72ed4964d5/payload-processing/0.log" Aug 04 10:03:16.491913 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:16.491855 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:03:16.822557 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:03:16.822473 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:15.315383 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.315352 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw_39ad9fb0-32ad-4209-a565-3d72ed4964d5/payload-processing/0.log" Aug 04 10:04:15.623774 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.623689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:15.959527 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.959490 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k"] Aug 04 10:04:15.963054 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.963035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:15.966526 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.966496 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-e0d739-openshift-default-dockercfg-sgsmc\"" Aug 04 10:04:15.987825 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.987800 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k"] Aug 04 10:04:15.996077 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:15.996054 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:16.000290 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61cdfec5-2068-485b-a6b2-dae164038933-istio-podinfo\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000319 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-certs\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61cdfec5-2068-485b-a6b2-dae164038933-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000380 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-data\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000440 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-istio-token\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-socket\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-credential-socket\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-envoy\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.000690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.000681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svftf\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-kube-api-access-svftf\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101334 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61cdfec5-2068-485b-a6b2-dae164038933-istio-podinfo\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101512 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101339 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-certs\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101512 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61cdfec5-2068-485b-a6b2-dae164038933-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101512 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-data\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101512 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-istio-token\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101710 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-socket\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101710 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-credential-socket\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101710 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-envoy\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101710 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svftf\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-kube-api-access-svftf\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101931 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101782 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-certs\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101931 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-data\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.101931 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.101894 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-socket\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.102064 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.102034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-credential-socket\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.102115 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.102096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61cdfec5-2068-485b-a6b2-dae164038933-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.103818 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.103793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-envoy\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.103956 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.103923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61cdfec5-2068-485b-a6b2-dae164038933-istio-podinfo\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.111199 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.111180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-istio-token\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.113103 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.113083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svftf\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-kube-api-access-svftf\") pod \"e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.183786 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.183756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:16.273140 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.273059 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:16.416339 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.416311 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k"] Aug 04 10:04:16.417629 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:04:16.417603 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cdfec5_2068_485b_a6b2_dae164038933.slice/crio-d586819cb46db288d1629c9e03caf72a0817f4c5978e9811ce4df073a839d4e0 WatchSource:0}: Error finding container d586819cb46db288d1629c9e03caf72a0817f4c5978e9811ce4df073a839d4e0: Status 404 returned error can't find the container with id d586819cb46db288d1629c9e03caf72a0817f4c5978e9811ce4df073a839d4e0 Aug 04 10:04:16.419744 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.419712 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:04:16.419822 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.419772 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:04:16.419822 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.419800 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:04:16.826772 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.826736 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" event={"ID":"61cdfec5-2068-485b-a6b2-dae164038933","Type":"ContainerStarted","Data":"26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621"} Aug 04 10:04:16.826941 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.826806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" event={"ID":"61cdfec5-2068-485b-a6b2-dae164038933","Type":"ContainerStarted","Data":"d586819cb46db288d1629c9e03caf72a0817f4c5978e9811ce4df073a839d4e0"} Aug 04 10:04:16.858138 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:16.858094 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podStartSLOduration=1.85807809 podStartE2EDuration="1.85807809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:04:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:04:16.856357039 +0000 UTC m=+3373.988566572" watchObservedRunningTime="2026-08-04 10:04:16.85807809 +0000 UTC m=+3373.990287675" Aug 04 10:04:17.273336 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:17.273302 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:17.278608 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:17.278581 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:17.832413 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:17.832388 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:17.833195 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:17.833178 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:18.840676 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:18.840639 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:18.940315 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:18.940285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:20.157934 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:20.157900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:20.670810 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:20.670769 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:20.815105 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:20.815074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:20.932974 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:20.932891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:33.874409 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:33.874365 2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Aug 04 10:04:33.874778 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:33.874424 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Aug 04 10:04:35.939209 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:35.939176 2562 generic.go:356] "Generic (PLEG): container finished" podID="61cdfec5-2068-485b-a6b2-dae164038933" containerID="26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621" exitCode=0 Aug 04 10:04:35.939655 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:35.939259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" event={"ID":"61cdfec5-2068-485b-a6b2-dae164038933","Type":"ContainerDied","Data":"26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621"} Aug 04 10:04:35.939655 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:35.939640 2562 scope.go:117] "RemoveContainer" containerID="26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621" Aug 04 10:04:35.940845 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:35.940821 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:04:35.940900 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:35.940887 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:04:35.940936 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:35.940917 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:04:36.952428 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:36.952387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" event={"ID":"61cdfec5-2068-485b-a6b2-dae164038933","Type":"ContainerStarted","Data":"7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19"} Aug 04 10:04:37.273885 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:37.273788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:37.274090 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:37.274070 2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Aug 04 10:04:37.274191 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:37.274114 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Aug 04 10:04:38.274271 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:38.274209 2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Aug 04 10:04:38.274652 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:38.274298 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Aug 04 10:04:39.274412 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:39.274373 2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Aug 04 10:04:39.274794 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:39.274438 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Aug 04 10:04:40.274455 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:40.274414 2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Aug 04 10:04:40.274844 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:40.274472 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Aug 04 10:04:41.274176 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:41.274138 2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body= Aug 04 10:04:41.274383 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:41.274193 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" Aug 04 10:04:42.276721 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:42.276682 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:42.277196 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:42.277049 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:42.277660 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:42.277643 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:04:43.867632 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:43.867598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:50.556422 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:50.556389 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:50.644622 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:50.644586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:50.712042 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:50.712010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:50.795404 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:50.795369 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:51.147984 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:51.147949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:57.379544 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:57.379466 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k"] Aug 04 10:04:57.379949 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:57.379724 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" containerID="cri-o://7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19" gracePeriod=30 Aug 04 10:04:57.448694 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:57.448664 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:57.575020 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:57.574968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:57.698128 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:04:57.698095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:02.633422 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.633399 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:05:02.661076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661003 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-socket\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661080 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svftf\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-kube-api-access-svftf\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661105 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-data\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661174 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-istio-token\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61cdfec5-2068-485b-a6b2-dae164038933-istio-podinfo\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661221 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-envoy\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661538 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661273 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-certs\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661538 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661309 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61cdfec5-2068-485b-a6b2-dae164038933-istiod-ca-cert\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661538 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661379 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-credential-socket\") pod \"61cdfec5-2068-485b-a6b2-dae164038933\" (UID: \"61cdfec5-2068-485b-a6b2-dae164038933\") " Aug 04 10:05:02.661538 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661478 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-data" (OuterVolumeSpecName: "istio-data") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:05:02.661740 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661696 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:05:02.661740 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661711 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:05:02.662312 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.661925 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:05:02.662312 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.662027 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61cdfec5-2068-485b-a6b2-dae164038933-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:05:02.662312 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.662053 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.662312 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.662216 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.662312 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.662234 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.662312 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.662270 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.664662 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.664627 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-kube-api-access-svftf" (OuterVolumeSpecName: "kube-api-access-svftf") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "kube-api-access-svftf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:05:02.664841 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.664804 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-istio-token" (OuterVolumeSpecName: "istio-token") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:05:02.665660 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.665635 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:05:02.665875 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.665853 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61cdfec5-2068-485b-a6b2-dae164038933-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61cdfec5-2068-485b-a6b2-dae164038933" (UID: "61cdfec5-2068-485b-a6b2-dae164038933"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 10:05:02.763527 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.763489 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svftf\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-kube-api-access-svftf\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.763527 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.763522 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61cdfec5-2068-485b-a6b2-dae164038933-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.763527 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.763537 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61cdfec5-2068-485b-a6b2-dae164038933-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.763756 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.763548 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61cdfec5-2068-485b-a6b2-dae164038933-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:02.763756 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:02.763560 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61cdfec5-2068-485b-a6b2-dae164038933-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:03.107473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.107382 2562 generic.go:356] "Generic (PLEG): container finished" podID="61cdfec5-2068-485b-a6b2-dae164038933" containerID="7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19" exitCode=0 Aug 04 10:05:03.107473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.107426 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" Aug 04 10:05:03.107721 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.107465 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" event={"ID":"61cdfec5-2068-485b-a6b2-dae164038933","Type":"ContainerDied","Data":"7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19"} Aug 04 10:05:03.107721 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.107525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k" event={"ID":"61cdfec5-2068-485b-a6b2-dae164038933","Type":"ContainerDied","Data":"d586819cb46db288d1629c9e03caf72a0817f4c5978e9811ce4df073a839d4e0"} Aug 04 10:05:03.107721 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.107554 2562 scope.go:117] "RemoveContainer" containerID="7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19" Aug 04 10:05:03.117902 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.117887 2562 scope.go:117] "RemoveContainer" containerID="26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621" Aug 04 10:05:03.126231 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.126215 2562 scope.go:117] "RemoveContainer" containerID="7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19" Aug 04 10:05:03.126521 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:05:03.126503 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19\": container with ID starting with 7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19 not found: ID does not exist" containerID="7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19" Aug 04 10:05:03.126567 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.126529 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19"} err="failed to get container status \"7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19\": rpc error: code = NotFound desc = could not find container \"7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19\": container with ID starting with 7dd24c5db8f1820e52d8957d93382baa8ed85baa30b2506815f25e155bce4f19 not found: ID does not exist" Aug 04 10:05:03.126567 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.126546 2562 scope.go:117] "RemoveContainer" containerID="26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621" Aug 04 10:05:03.126774 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:05:03.126757 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621\": container with ID starting with 26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621 not found: ID does not exist" containerID="26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621" Aug 04 10:05:03.126814 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.126781 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621"} err="failed to get container status \"26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621\": rpc error: code = NotFound desc = could not find container \"26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621\": container with ID starting with 26ee4c95ee5fcbd0269662bd525a523f2d27d447a24d3cb227ca679edafb9621 not found: ID does not exist" Aug 04 10:05:03.133873 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.133847 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k"] Aug 04 10:05:03.136412 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.136388 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e0d739-openshift-default-6798f658dd-7v56k"] Aug 04 10:05:03.417067 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:03.417035 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61cdfec5-2068-485b-a6b2-dae164038933" path="/var/lib/kubelet/pods/61cdfec5-2068-485b-a6b2-dae164038933/volumes" Aug 04 10:05:16.742674 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:16.742637 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2"] Aug 04 10:05:16.743050 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:16.742903 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" podUID="7a8972da-b2a7-4df0-8873-4f9b68b68026" containerName="payload-pre-processing" containerID="cri-o://8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293" gracePeriod=30 Aug 04 10:05:16.907080 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:16.907044 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:16.982372 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:16.982156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:16.998555 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:16.998502 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:05:17.103075 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.103030 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a8972da-b2a7-4df0-8873-4f9b68b68026-plugins-config-volume\") pod \"7a8972da-b2a7-4df0-8873-4f9b68b68026\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " Aug 04 10:05:17.103247 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.103103 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf242\" (UniqueName: \"kubernetes.io/projected/7a8972da-b2a7-4df0-8873-4f9b68b68026-kube-api-access-pf242\") pod \"7a8972da-b2a7-4df0-8873-4f9b68b68026\" (UID: \"7a8972da-b2a7-4df0-8873-4f9b68b68026\") " Aug 04 10:05:17.103442 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.103418 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8972da-b2a7-4df0-8873-4f9b68b68026-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a8972da-b2a7-4df0-8873-4f9b68b68026" (UID: "7a8972da-b2a7-4df0-8873-4f9b68b68026"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:05:17.105106 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.105084 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8972da-b2a7-4df0-8873-4f9b68b68026-kube-api-access-pf242" (OuterVolumeSpecName: "kube-api-access-pf242") pod "7a8972da-b2a7-4df0-8873-4f9b68b68026" (UID: "7a8972da-b2a7-4df0-8873-4f9b68b68026"). InnerVolumeSpecName "kube-api-access-pf242". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:05:17.189109 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.189075 2562 generic.go:356] "Generic (PLEG): container finished" podID="7a8972da-b2a7-4df0-8873-4f9b68b68026" containerID="8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293" exitCode=0 Aug 04 10:05:17.189308 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.189120 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" Aug 04 10:05:17.189308 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.189157 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" event={"ID":"7a8972da-b2a7-4df0-8873-4f9b68b68026","Type":"ContainerDied","Data":"8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293"} Aug 04 10:05:17.189308 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.189208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2" event={"ID":"7a8972da-b2a7-4df0-8873-4f9b68b68026","Type":"ContainerDied","Data":"3c81afc7131a74b4553ed5d91897e8c3da0ab3adf9ce773749f82a097f775c2f"} Aug 04 10:05:17.189308 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.189232 2562 scope.go:117] "RemoveContainer" containerID="8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293" Aug 04 10:05:17.198503 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.198483 2562 scope.go:117] "RemoveContainer" containerID="8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293" Aug 04 10:05:17.198738 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:05:17.198717 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293\": container with ID starting with 8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293 not found: ID does not exist" containerID="8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293" Aug 04 10:05:17.198812 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.198750 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293"} err="failed to get container status \"8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293\": rpc error: code = NotFound desc = could not find container \"8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293\": container with ID starting with 8e1371de50395b0a7761a13af61ac729faf27cc42ab9dc3e5a50ff92394c1293 not found: ID does not exist" Aug 04 10:05:17.204701 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.204680 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a8972da-b2a7-4df0-8873-4f9b68b68026-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:17.204701 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.204702 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf242\" (UniqueName: \"kubernetes.io/projected/7a8972da-b2a7-4df0-8873-4f9b68b68026-kube-api-access-pf242\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:05:17.230591 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.230565 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2"] Aug 04 10:05:17.266631 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.266569 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fc7b93-6f967fdb4d-sg2h2"] Aug 04 10:05:17.351631 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.351596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:17.416845 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:17.416808 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a8972da-b2a7-4df0-8873-4f9b68b68026" path="/var/lib/kubelet/pods/7a8972da-b2a7-4df0-8873-4f9b68b68026/volumes" Aug 04 10:05:22.413610 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:22.413576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:27.230010 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:27.229979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:27.332188 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:27.332156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:31.483050 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:31.483015 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:58.926371 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:05:58.926335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:00.728803 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:00.728767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:06.613078 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:06.613042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:06.725182 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:06.725147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:06.913957 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:06.913920 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8"] Aug 04 10:06:06.914160 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:06.914137 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" containerID="cri-o://76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb" gracePeriod=30 Aug 04 10:06:06.944288 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:06.944240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:07.212905 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:07.212826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:12.160313 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.160285 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:06:12.225243 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225166 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-certs\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225243 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225223 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-socket\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225488 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225283 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-podinfo\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225488 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225341 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2mkv\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-kube-api-access-s2mkv\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225488 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225372 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-credential-socket\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225657 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225460 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:12.225657 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225501 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istiod-ca-cert\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225657 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225527 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:12.225657 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225578 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-envoy\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225657 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225595 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-token\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225657 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225615 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-data\") pod \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\" (UID: \"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6\") " Aug 04 10:06:12.225969 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225664 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:12.225969 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225819 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:06:12.225969 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.225953 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:12.226131 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.226108 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.226196 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.226135 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.226196 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.226149 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.226196 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.226161 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.226196 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.226173 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.227794 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.227764 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:12.227918 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.227837 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-kube-api-access-s2mkv" (OuterVolumeSpecName: "kube-api-access-s2mkv") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "kube-api-access-s2mkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:12.227967 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.227943 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:12.228069 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.228050 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" (UID: "ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 10:06:12.327523 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.327494 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.327523 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.327519 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2mkv\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-kube-api-access-s2mkv\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.327523 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.327529 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.327725 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.327537 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:12.516130 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.516045 2562 generic.go:356] "Generic (PLEG): container finished" podID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerID="76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb" exitCode=0 Aug 04 10:06:12.516130 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.516103 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" Aug 04 10:06:12.516334 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.516125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" event={"ID":"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6","Type":"ContainerDied","Data":"76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb"} Aug 04 10:06:12.516334 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.516177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8" event={"ID":"ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6","Type":"ContainerDied","Data":"e8b72bb6bdd8992b871cdc81c4d9a77d09d12843f1daa84a713afe9ad7203f00"} Aug 04 10:06:12.516334 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.516200 2562 scope.go:117] "RemoveContainer" containerID="76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb" Aug 04 10:06:12.526695 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.526678 2562 scope.go:117] "RemoveContainer" containerID="66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa" Aug 04 10:06:12.534996 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.534979 2562 scope.go:117] "RemoveContainer" containerID="76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb" Aug 04 10:06:12.535286 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:12.535267 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb\": container with ID starting with 76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb not found: ID does not exist" containerID="76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb" Aug 04 10:06:12.535342 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.535296 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb"} err="failed to get container status \"76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb\": rpc error: code = NotFound desc = could not find container \"76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb\": container with ID starting with 76be8bcf5bb14e8cbd93aff54498705f83768ba105ab2dc55d8578b6050fa7cb not found: ID does not exist" Aug 04 10:06:12.535342 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.535314 2562 scope.go:117] "RemoveContainer" containerID="66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa" Aug 04 10:06:12.535537 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:12.535518 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa\": container with ID starting with 66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa not found: ID does not exist" containerID="66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa" Aug 04 10:06:12.535588 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.535543 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa"} err="failed to get container status \"66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa\": rpc error: code = NotFound desc = could not find container \"66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa\": container with ID starting with 66d38d4087e4582a8dfa7043e3b6d897e3f7a7470fa79d05934b0ce67314cfaa not found: ID does not exist" Aug 04 10:06:12.541460 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.541438 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8"] Aug 04 10:06:12.548570 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:12.548544 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-fc7b93-openshift-default-7849cb6b5d-xffq8"] Aug 04 10:06:13.416745 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:13.416711 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" path="/var/lib/kubelet/pods/ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6/volumes" Aug 04 10:06:17.936033 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:17.935998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:17.973651 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:17.973619 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw"] Aug 04 10:06:17.973946 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:17.973911 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" podUID="39ad9fb0-32ad-4209-a565-3d72ed4964d5" containerName="payload-processing" containerID="cri-o://19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da" gracePeriod=30 Aug 04 10:06:17.985838 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:17.985818 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9"] Aug 04 10:06:17.986067 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:17.986042 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" podUID="36b78755-0836-46d8-9bae-8ffb47f2d406" containerName="payload-pre-processing" containerID="cri-o://ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9" gracePeriod=30 Aug 04 10:06:18.027533 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.027507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:18.123474 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.123439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:18.204179 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.204144 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:18.259936 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.259914 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:06:18.262800 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.262780 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:06:18.382197 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382170 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b78755-0836-46d8-9bae-8ffb47f2d406-plugins-config-volume\") pod \"36b78755-0836-46d8-9bae-8ffb47f2d406\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " Aug 04 10:06:18.382384 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382288 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pftbh\" (UniqueName: \"kubernetes.io/projected/36b78755-0836-46d8-9bae-8ffb47f2d406-kube-api-access-pftbh\") pod \"36b78755-0836-46d8-9bae-8ffb47f2d406\" (UID: \"36b78755-0836-46d8-9bae-8ffb47f2d406\") " Aug 04 10:06:18.382384 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382317 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzhs9\" (UniqueName: \"kubernetes.io/projected/39ad9fb0-32ad-4209-a565-3d72ed4964d5-kube-api-access-rzhs9\") pod \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " Aug 04 10:06:18.382384 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382345 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ad9fb0-32ad-4209-a565-3d72ed4964d5-plugins-config-volume\") pod \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\" (UID: \"39ad9fb0-32ad-4209-a565-3d72ed4964d5\") " Aug 04 10:06:18.382583 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382562 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b78755-0836-46d8-9bae-8ffb47f2d406-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36b78755-0836-46d8-9bae-8ffb47f2d406" (UID: "36b78755-0836-46d8-9bae-8ffb47f2d406"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:06:18.382731 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382710 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b78755-0836-46d8-9bae-8ffb47f2d406-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:18.382822 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.382801 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ad9fb0-32ad-4209-a565-3d72ed4964d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39ad9fb0-32ad-4209-a565-3d72ed4964d5" (UID: "39ad9fb0-32ad-4209-a565-3d72ed4964d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:06:18.384514 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.384483 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ad9fb0-32ad-4209-a565-3d72ed4964d5-kube-api-access-rzhs9" (OuterVolumeSpecName: "kube-api-access-rzhs9") pod "39ad9fb0-32ad-4209-a565-3d72ed4964d5" (UID: "39ad9fb0-32ad-4209-a565-3d72ed4964d5"). InnerVolumeSpecName "kube-api-access-rzhs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:18.384514 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.384510 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b78755-0836-46d8-9bae-8ffb47f2d406-kube-api-access-pftbh" (OuterVolumeSpecName: "kube-api-access-pftbh") pod "36b78755-0836-46d8-9bae-8ffb47f2d406" (UID: "36b78755-0836-46d8-9bae-8ffb47f2d406"). InnerVolumeSpecName "kube-api-access-pftbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:18.483320 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.483221 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pftbh\" (UniqueName: \"kubernetes.io/projected/36b78755-0836-46d8-9bae-8ffb47f2d406-kube-api-access-pftbh\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:18.483320 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.483272 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzhs9\" (UniqueName: \"kubernetes.io/projected/39ad9fb0-32ad-4209-a565-3d72ed4964d5-kube-api-access-rzhs9\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:18.483320 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.483282 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39ad9fb0-32ad-4209-a565-3d72ed4964d5-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:18.557093 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.557057 2562 generic.go:356] "Generic (PLEG): container finished" podID="39ad9fb0-32ad-4209-a565-3d72ed4964d5" containerID="19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da" exitCode=0 Aug 04 10:06:18.557243 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.557101 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" Aug 04 10:06:18.557243 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.557137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" event={"ID":"39ad9fb0-32ad-4209-a565-3d72ed4964d5","Type":"ContainerDied","Data":"19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da"} Aug 04 10:06:18.557243 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.557191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw" event={"ID":"39ad9fb0-32ad-4209-a565-3d72ed4964d5","Type":"ContainerDied","Data":"125bd3d3c2319b61892be63070fd55f49e0153c6b0d801067c90fb4eb88416d5"} Aug 04 10:06:18.557243 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.557224 2562 scope.go:117] "RemoveContainer" containerID="19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da" Aug 04 10:06:18.558613 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.558580 2562 generic.go:356] "Generic (PLEG): container finished" podID="36b78755-0836-46d8-9bae-8ffb47f2d406" containerID="ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9" exitCode=0 Aug 04 10:06:18.558735 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.558621 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" Aug 04 10:06:18.558735 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.558627 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" event={"ID":"36b78755-0836-46d8-9bae-8ffb47f2d406","Type":"ContainerDied","Data":"ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9"} Aug 04 10:06:18.558735 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.558673 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9" event={"ID":"36b78755-0836-46d8-9bae-8ffb47f2d406","Type":"ContainerDied","Data":"e0080d5704f4c00e942b4b00555a2834fefe915f5674ccdbcece5bb18f32d959"} Aug 04 10:06:18.566816 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.566605 2562 scope.go:117] "RemoveContainer" containerID="19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da" Aug 04 10:06:18.567013 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:18.566994 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da\": container with ID starting with 19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da not found: ID does not exist" containerID="19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da" Aug 04 10:06:18.567081 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.567021 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da"} err="failed to get container status \"19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da\": rpc error: code = NotFound desc = could not find container \"19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da\": container with ID starting with 19763e152ecbfa6f9e168254b1a8dc5d227d8af3bccba21d9691f64b6e2d33da not found: ID does not exist" Aug 04 10:06:18.567081 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.567039 2562 scope.go:117] "RemoveContainer" containerID="ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9" Aug 04 10:06:18.571306 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.571285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:18.575557 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.575543 2562 scope.go:117] "RemoveContainer" containerID="ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9" Aug 04 10:06:18.575812 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:18.575793 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9\": container with ID starting with ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9 not found: ID does not exist" containerID="ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9" Aug 04 10:06:18.575862 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.575817 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9"} err="failed to get container status \"ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9\": rpc error: code = NotFound desc = could not find container \"ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9\": container with ID starting with ddd4c5b4f68054dc1adc6a98736ba8cd73b0fa10273b201fe8d0622d538ce5f9 not found: ID does not exist" Aug 04 10:06:18.586210 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.586189 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw"] Aug 04 10:06:18.591131 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.591111 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-90d908-7fdddddb48-pjdhw"] Aug 04 10:06:18.601770 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.601752 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9"] Aug 04 10:06:18.608994 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:18.608973 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-90d908-57c8dbd9fb-4pfg9"] Aug 04 10:06:19.417319 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:19.417286 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b78755-0836-46d8-9bae-8ffb47f2d406" path="/var/lib/kubelet/pods/36b78755-0836-46d8-9bae-8ffb47f2d406/volumes" Aug 04 10:06:19.417688 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:19.417663 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ad9fb0-32ad-4209-a565-3d72ed4964d5" path="/var/lib/kubelet/pods/39ad9fb0-32ad-4209-a565-3d72ed4964d5/volumes" Aug 04 10:06:24.445307 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:24.445275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:24.549412 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:24.549382 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:24.742827 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:24.742717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:25.004176 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:25.004067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:35.744472 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:35.744437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:35.775583 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:35.775548 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp"] Aug 04 10:06:35.775822 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:35.775794 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" podUID="c25bb48f-0b54-422d-9d24-9808f67f8980" containerName="payload-processing" containerID="cri-o://e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80" gracePeriod=30 Aug 04 10:06:35.822231 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:35.822196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:35.927386 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:35.927354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:35.995618 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:35.995539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:36.032829 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.032808 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 10:06:36.148387 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.148339 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c25bb48f-0b54-422d-9d24-9808f67f8980-plugins-config-volume\") pod \"c25bb48f-0b54-422d-9d24-9808f67f8980\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " Aug 04 10:06:36.148387 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.148397 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qkjr\" (UniqueName: \"kubernetes.io/projected/c25bb48f-0b54-422d-9d24-9808f67f8980-kube-api-access-6qkjr\") pod \"c25bb48f-0b54-422d-9d24-9808f67f8980\" (UID: \"c25bb48f-0b54-422d-9d24-9808f67f8980\") " Aug 04 10:06:36.148763 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.148742 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c25bb48f-0b54-422d-9d24-9808f67f8980-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c25bb48f-0b54-422d-9d24-9808f67f8980" (UID: "c25bb48f-0b54-422d-9d24-9808f67f8980"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:06:36.150411 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.150389 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25bb48f-0b54-422d-9d24-9808f67f8980-kube-api-access-6qkjr" (OuterVolumeSpecName: "kube-api-access-6qkjr") pod "c25bb48f-0b54-422d-9d24-9808f67f8980" (UID: "c25bb48f-0b54-422d-9d24-9808f67f8980"). InnerVolumeSpecName "kube-api-access-6qkjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:36.249463 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.249383 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c25bb48f-0b54-422d-9d24-9808f67f8980-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:36.249463 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.249411 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qkjr\" (UniqueName: \"kubernetes.io/projected/c25bb48f-0b54-422d-9d24-9808f67f8980-kube-api-access-6qkjr\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:36.343001 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.342965 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:36.666311 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.666273 2562 generic.go:356] "Generic (PLEG): container finished" podID="c25bb48f-0b54-422d-9d24-9808f67f8980" containerID="e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80" exitCode=0 Aug 04 10:06:36.666486 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.666316 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" Aug 04 10:06:36.666486 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.666355 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" event={"ID":"c25bb48f-0b54-422d-9d24-9808f67f8980","Type":"ContainerDied","Data":"e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80"} Aug 04 10:06:36.666486 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.666412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp" event={"ID":"c25bb48f-0b54-422d-9d24-9808f67f8980","Type":"ContainerDied","Data":"d3e43a8f8ef96b6621c86e473b325b6a964fb2017674babd69571798d35f5971"} Aug 04 10:06:36.666486 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.666436 2562 scope.go:117] "RemoveContainer" containerID="e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80" Aug 04 10:06:36.675561 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.675543 2562 scope.go:117] "RemoveContainer" containerID="e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80" Aug 04 10:06:36.675786 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:36.675767 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80\": container with ID starting with e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80 not found: ID does not exist" containerID="e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80" Aug 04 10:06:36.675838 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.675795 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80"} err="failed to get container status \"e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80\": rpc error: code = NotFound desc = could not find container \"e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80\": container with ID starting with e0247eee5ede794c34b599c050d973f1c25749740c9d587506e50450a3751b80 not found: ID does not exist" Aug 04 10:06:36.692778 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.692752 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp"] Aug 04 10:06:36.700967 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:36.700941 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-497679-6c8d4654d6-78ktp"] Aug 04 10:06:37.416745 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:37.416711 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c25bb48f-0b54-422d-9d24-9808f67f8980" path="/var/lib/kubelet/pods/c25bb48f-0b54-422d-9d24-9808f67f8980/volumes" Aug 04 10:06:42.700744 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:42.700705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:42.805676 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:42.805640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:42.988808 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:42.988729 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx"] Aug 04 10:06:42.989004 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:42.988979 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" containerID="cri-o://ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137" gracePeriod=30 Aug 04 10:06:43.065349 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:43.065313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:43.244279 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:43.244170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:46.122520 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:46.122484 2562 patch_prober.go:28] interesting pod/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 10:06:46.122908 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:46.122541 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 10:06:48.242188 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.242163 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 10:06:48.253733 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253677 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c403c916-d126-40ac-83b6-68a84957e9e6-istiod-ca-cert\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.253733 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253722 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv2rk\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-kube-api-access-tv2rk\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.253870 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253743 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-socket\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.253870 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253815 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-credential-socket\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.253870 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253842 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c403c916-d126-40ac-83b6-68a84957e9e6-istio-podinfo\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.254025 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253882 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-envoy\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.254025 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253917 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-certs\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.254025 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253950 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-istio-token\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.254025 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.253976 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-data\") pod \"c403c916-d126-40ac-83b6-68a84957e9e6\" (UID: \"c403c916-d126-40ac-83b6-68a84957e9e6\") " Aug 04 10:06:48.254224 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254017 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c403c916-d126-40ac-83b6-68a84957e9e6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:06:48.254321 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254216 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:48.254377 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254329 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:48.254377 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254359 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-certs\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.254482 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254388 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c403c916-d126-40ac-83b6-68a84957e9e6-istiod-ca-cert\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.254482 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254393 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:48.254482 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.254415 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-data" (OuterVolumeSpecName: "istio-data") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:48.256093 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.256061 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c403c916-d126-40ac-83b6-68a84957e9e6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 10:06:48.256093 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.256075 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:48.256093 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.256073 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:48.256416 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.256400 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-kube-api-access-tv2rk" (OuterVolumeSpecName: "kube-api-access-tv2rk") pod "c403c916-d126-40ac-83b6-68a84957e9e6" (UID: "c403c916-d126-40ac-83b6-68a84957e9e6"). InnerVolumeSpecName "kube-api-access-tv2rk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:48.354961 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354930 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-credential-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.354961 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354956 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c403c916-d126-40ac-83b6-68a84957e9e6-istio-podinfo\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.354961 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354965 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-envoy\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.355175 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354974 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-istio-token\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.355175 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354982 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-istio-data\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.355175 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354989 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv2rk\" (UniqueName: \"kubernetes.io/projected/c403c916-d126-40ac-83b6-68a84957e9e6-kube-api-access-tv2rk\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.355175 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.354998 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c403c916-d126-40ac-83b6-68a84957e9e6-workload-socket\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:48.735713 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.735680 2562 generic.go:356] "Generic (PLEG): container finished" podID="c403c916-d126-40ac-83b6-68a84957e9e6" containerID="ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137" exitCode=0 Aug 04 10:06:48.735875 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.735731 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" Aug 04 10:06:48.735875 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.735770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" event={"ID":"c403c916-d126-40ac-83b6-68a84957e9e6","Type":"ContainerDied","Data":"ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137"} Aug 04 10:06:48.735875 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.735822 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx" event={"ID":"c403c916-d126-40ac-83b6-68a84957e9e6","Type":"ContainerDied","Data":"e6057098461f741a7ffa9bb0ce867a99f2e346a5d6285f8d91d287d8a8cd34c9"} Aug 04 10:06:48.735875 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.735844 2562 scope.go:117] "RemoveContainer" containerID="ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137" Aug 04 10:06:48.745436 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.745409 2562 scope.go:117] "RemoveContainer" containerID="79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7" Aug 04 10:06:48.753488 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.753473 2562 scope.go:117] "RemoveContainer" containerID="ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137" Aug 04 10:06:48.753736 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:48.753706 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137\": container with ID starting with ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137 not found: ID does not exist" containerID="ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137" Aug 04 10:06:48.753818 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.753743 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137"} err="failed to get container status \"ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137\": rpc error: code = NotFound desc = could not find container \"ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137\": container with ID starting with ac28d2808ec94d69016b1e99778f44f472a221283789c5558825224c3431d137 not found: ID does not exist" Aug 04 10:06:48.753818 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.753759 2562 scope.go:117] "RemoveContainer" containerID="79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7" Aug 04 10:06:48.754002 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:48.753984 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7\": container with ID starting with 79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7 not found: ID does not exist" containerID="79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7" Aug 04 10:06:48.754040 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.754008 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7"} err="failed to get container status \"79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7\": rpc error: code = NotFound desc = could not find container \"79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7\": container with ID starting with 79514a7da664f3ab539c17d9021e03897036686a0e0f5c1006f483eae6b8c7a7 not found: ID does not exist" Aug 04 10:06:48.761341 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.760500 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx"] Aug 04 10:06:48.769722 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:48.769683 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-497679-openshift-default-7d9575b58d-f8bmx"] Aug 04 10:06:49.417476 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:49.417440 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" path="/var/lib/kubelet/pods/c403c916-d126-40ac-83b6-68a84957e9e6/volumes" Aug 04 10:06:53.919644 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:53.919610 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:53.977076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:53.977038 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr"] Aug 04 10:06:53.977336 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:53.977311 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" podUID="f1ab44bf-0de5-4765-8387-44174042b534" containerName="payload-pre-processing" containerID="cri-o://bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b" gracePeriod=30 Aug 04 10:06:54.017875 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.017848 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:54.122496 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.122468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:54.204153 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.204084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:54.242511 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.238771 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 10:06:54.300812 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.300777 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxcsd\" (UniqueName: \"kubernetes.io/projected/f1ab44bf-0de5-4765-8387-44174042b534-kube-api-access-hxcsd\") pod \"f1ab44bf-0de5-4765-8387-44174042b534\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " Aug 04 10:06:54.300968 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.300836 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ab44bf-0de5-4765-8387-44174042b534-plugins-config-volume\") pod \"f1ab44bf-0de5-4765-8387-44174042b534\" (UID: \"f1ab44bf-0de5-4765-8387-44174042b534\") " Aug 04 10:06:54.301241 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.301219 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ab44bf-0de5-4765-8387-44174042b534-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1ab44bf-0de5-4765-8387-44174042b534" (UID: "f1ab44bf-0de5-4765-8387-44174042b534"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:06:54.302915 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.302891 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ab44bf-0de5-4765-8387-44174042b534-kube-api-access-hxcsd" (OuterVolumeSpecName: "kube-api-access-hxcsd") pod "f1ab44bf-0de5-4765-8387-44174042b534" (UID: "f1ab44bf-0de5-4765-8387-44174042b534"). InnerVolumeSpecName "kube-api-access-hxcsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:54.401594 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.401563 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxcsd\" (UniqueName: \"kubernetes.io/projected/f1ab44bf-0de5-4765-8387-44174042b534-kube-api-access-hxcsd\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:54.401594 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.401587 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ab44bf-0de5-4765-8387-44174042b534-plugins-config-volume\") on node \"ip-10-0-136-20.ec2.internal\" DevicePath \"\"" Aug 04 10:06:54.607460 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.607376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:06:54.773054 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.773021 2562 generic.go:356] "Generic (PLEG): container finished" podID="f1ab44bf-0de5-4765-8387-44174042b534" containerID="bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b" exitCode=0 Aug 04 10:06:54.773225 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.773071 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" Aug 04 10:06:54.773225 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.773105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" event={"ID":"f1ab44bf-0de5-4765-8387-44174042b534","Type":"ContainerDied","Data":"bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b"} Aug 04 10:06:54.773225 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.773161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr" event={"ID":"f1ab44bf-0de5-4765-8387-44174042b534","Type":"ContainerDied","Data":"dc2e151d186a640d7a88b1f43b505d42bfefec6ffda7c22a4c192bd5df131ecf"} Aug 04 10:06:54.773225 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.773183 2562 scope.go:117] "RemoveContainer" containerID="bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b" Aug 04 10:06:54.782208 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.782191 2562 scope.go:117] "RemoveContainer" containerID="bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b" Aug 04 10:06:54.782511 ip-10-0-136-20 kubenswrapper[2562]: E0804 10:06:54.782491 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b\": container with ID starting with bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b not found: ID does not exist" containerID="bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b" Aug 04 10:06:54.782590 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.782520 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b"} err="failed to get container status \"bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b\": rpc error: code = NotFound desc = could not find container \"bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b\": container with ID starting with bf12b87e7e65d9424c842ccbbb46b198e37df8845d265c13ee3c911b0274f71b not found: ID does not exist" Aug 04 10:06:54.797033 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.797010 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr"] Aug 04 10:06:54.803850 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:54.803830 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-b0bc81-f64d665b8-fc6lr"] Aug 04 10:06:55.416492 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:06:55.416460 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ab44bf-0de5-4765-8387-44174042b534" path="/var/lib/kubelet/pods/f1ab44bf-0de5-4765-8387-44174042b534/volumes" Aug 04 10:07:00.985780 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:00.985747 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:01.069609 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:01.069532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:01.282641 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:01.282559 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:01.502274 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:01.502199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:01.913478 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:01.913450 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56c4678dcd-f8s65_a42398c4-7076-49d1-aec3-bd0a03add1e6/authorino/0.log" Aug 04 10:07:05.632324 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:05.632294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:07.962648 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:07.962619 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xqm6t_115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6/server/0.log" Aug 04 10:07:08.085431 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:08.085402 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-94szl_b2f2546a-5a85-4f91-ad18-6d404200fd5c/manager/0.log" Aug 04 10:07:08.198888 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:08.198860 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-bzzrg_9182437b-86c5-4255-8e0c-fa5a01d8b05a/copy-manifests/0.log" Aug 04 10:07:08.205754 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:08.205727 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-bzzrg_9182437b-86c5-4255-8e0c-fa5a01d8b05a/manager/0.log" Aug 04 10:07:08.942826 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:08.942798 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29763930-2wdws_e0e889f0-d4c9-47b4-924e-a39c03e9ea6c/cleanup/0.log" Aug 04 10:07:09.060793 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:09.060767 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29763945-ctt5p_2260c067-fc66-4ccc-8ed0-189428a437d4/cleanup/0.log" Aug 04 10:07:10.115168 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:10.115140 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56c4678dcd-f8s65_a42398c4-7076-49d1-aec3-bd0a03add1e6/authorino/0.log" Aug 04 10:07:11.182413 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:11.182382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99_5e958605-71b9-4c12-946c-a38e22ddb5c6/istio-proxy/0.log" Aug 04 10:07:11.520285 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:11.520204 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-xnn4z_3ebe8684-4d19-4144-96d7-a66c6fdb78e7/kube-auth-proxy/0.log" Aug 04 10:07:11.754269 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:11.754222 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ncr4r_d43c1705-8f57-412d-bd66-cf9b141ece3b/payload-pre-processing/0.log" Aug 04 10:07:12.711666 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:12.711637 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmltj5_c33ac7ab-cd4d-4844-891d-f0b80687f3d0/main/0.log" Aug 04 10:07:18.719466 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719432 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ddhlv/must-gather-whhxm"] Aug 04 10:07:18.719878 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719863 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25bb48f-0b54-422d-9d24-9808f67f8980" containerName="payload-processing" Aug 04 10:07:18.719924 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719882 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25bb48f-0b54-422d-9d24-9808f67f8980" containerName="payload-processing" Aug 04 10:07:18.719924 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719896 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" Aug 04 10:07:18.719924 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719902 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" Aug 04 10:07:18.719924 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719909 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" Aug 04 10:07:18.719924 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719915 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719926 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719931 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719938 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719943 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719953 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ad9fb0-32ad-4209-a565-3d72ed4964d5" containerName="payload-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719958 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ad9fb0-32ad-4209-a565-3d72ed4964d5" containerName="payload-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719964 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ab44bf-0de5-4765-8387-44174042b534" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719969 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ab44bf-0de5-4765-8387-44174042b534" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719976 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b78755-0836-46d8-9bae-8ffb47f2d406" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719984 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b78755-0836-46d8-9bae-8ffb47f2d406" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.719997 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8972da-b2a7-4df0-8873-4f9b68b68026" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720003 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8972da-b2a7-4df0-8873-4f9b68b68026" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720064 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b78755-0836-46d8-9bae-8ffb47f2d406" containerName="payload-pre-processing" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720072 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" Aug 04 10:07:18.720076 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720079 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c25bb48f-0b54-422d-9d24-9808f67f8980" containerName="payload-processing" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720086 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8972da-b2a7-4df0-8873-4f9b68b68026" containerName="payload-pre-processing" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720094 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720100 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ad9fb0-32ad-4209-a565-3d72ed4964d5" containerName="payload-processing" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720106 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720112 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720120 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720129 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ab44bf-0de5-4765-8387-44174042b534" containerName="payload-pre-processing" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720194 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720199 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c403c916-d126-40ac-83b6-68a84957e9e6" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720205 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720210 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="61cdfec5-2068-485b-a6b2-dae164038933" containerName="istio-proxy" Aug 04 10:07:18.720546 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.720280 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe82bf0-b6c7-4a1d-9cd1-304eed0a16d6" containerName="istio-proxy" Aug 04 10:07:18.723350 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.723334 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:18.726386 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.726361 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddhlv\"/\"openshift-service-ca.crt\"" Aug 04 10:07:18.726386 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.726376 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddhlv\"/\"kube-root-ca.crt\"" Aug 04 10:07:18.727261 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.727232 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ddhlv\"/\"default-dockercfg-lb999\"" Aug 04 10:07:18.732022 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.731996 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:18.732995 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.732974 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddhlv/must-gather-whhxm"] Aug 04 10:07:18.819515 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.819487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad-must-gather-output\") pod \"must-gather-whhxm\" (UID: \"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad\") " pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:18.819677 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.819526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdqtc\" (UniqueName: \"kubernetes.io/projected/3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad-kube-api-access-pdqtc\") pod \"must-gather-whhxm\" (UID: \"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad\") " pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:18.849342 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.849314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:18.920088 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.920057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad-must-gather-output\") pod \"must-gather-whhxm\" (UID: \"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad\") " pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:18.920317 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.920110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdqtc\" (UniqueName: \"kubernetes.io/projected/3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad-kube-api-access-pdqtc\") pod \"must-gather-whhxm\" (UID: \"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad\") " pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:18.920460 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.920440 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad-must-gather-output\") pod \"must-gather-whhxm\" (UID: \"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad\") " pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:18.932049 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:18.932019 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdqtc\" (UniqueName: \"kubernetes.io/projected/3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad-kube-api-access-pdqtc\") pod \"must-gather-whhxm\" (UID: \"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad\") " pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:19.033316 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:19.033215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddhlv/must-gather-whhxm" Aug 04 10:07:19.169760 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:07:19.169728 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d8c31d6_d9dd_44ab_ba64_ee7a2abf53ad.slice/crio-5e4047236ca4bc8ef2860e112f11a906782645031af3335dc15489a78c700904 WatchSource:0}: Error finding container 5e4047236ca4bc8ef2860e112f11a906782645031af3335dc15489a78c700904: Status 404 returned error can't find the container with id 5e4047236ca4bc8ef2860e112f11a906782645031af3335dc15489a78c700904 Aug 04 10:07:19.171328 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:19.171298 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddhlv/must-gather-whhxm"] Aug 04 10:07:19.171892 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:19.171874 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:07:19.916736 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:19.916700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/must-gather-whhxm" event={"ID":"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad","Type":"ContainerStarted","Data":"5e4047236ca4bc8ef2860e112f11a906782645031af3335dc15489a78c700904"} Aug 04 10:07:20.927304 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:20.927240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/must-gather-whhxm" event={"ID":"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad","Type":"ContainerStarted","Data":"6719e088983e2794f452821fc9e336acb69b3d9b7562f7ea7925b1f22a8530d0"} Aug 04 10:07:20.927861 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:20.927835 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/must-gather-whhxm" event={"ID":"3d8c31d6-d9dd-44ab-ba64-ee7a2abf53ad","Type":"ContainerStarted","Data":"ef664fc4237b6467abf98a2ace7e9343e97bf8cfa97799d394d252784a826945"} Aug 04 10:07:20.947324 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:20.947244 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ddhlv/must-gather-whhxm" podStartSLOduration=1.9553891060000002 podStartE2EDuration="2.947227861s" totalImagesPullingTime="991.838755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:07:20.944841643 +0000 UTC m=+3558.077051175" watchObservedRunningTime="2026-08-04 10:07:20.947227861 +0000 UTC m=+3558.079437397" Aug 04 10:07:21.781211 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:21.781163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cnb48_bbbd5193-de5e-4b69-bb4c-94ce6f8c0ecc/global-pull-secret-syncer/0.log" Aug 04 10:07:21.889737 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:21.889710 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-22b68_dbdb2aba-45f2-408b-9c0d-30a4e561dd9f/konnectivity-agent/0.log" Aug 04 10:07:22.003096 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:22.003069 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-20.ec2.internal_2c069c649a5d879b02b53f674cdd665d/haproxy/0.log" Aug 04 10:07:26.776516 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:26.776481 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56c4678dcd-f8s65_a42398c4-7076-49d1-aec3-bd0a03add1e6/authorino/0.log" Aug 04 10:07:28.472542 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.472509 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/alertmanager/0.log" Aug 04 10:07:28.497535 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.497504 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/config-reloader/0.log" Aug 04 10:07:28.521613 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.521587 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/kube-rbac-proxy-web/0.log" Aug 04 10:07:28.547859 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.547826 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/kube-rbac-proxy/0.log" Aug 04 10:07:28.573573 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.573546 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/kube-rbac-proxy-metric/0.log" Aug 04 10:07:28.598146 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.598116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/prom-label-proxy/0.log" Aug 04 10:07:28.624015 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.623988 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45219cfe-41d1-4a96-933a-074bd8063e0b/init-config-reloader/0.log" Aug 04 10:07:28.871185 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.871105 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-j78v6_019cf2ad-7780-423b-a94f-2fd9badc4649/kube-state-metrics/0.log" Aug 04 10:07:28.892050 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.891988 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-j78v6_019cf2ad-7780-423b-a94f-2fd9badc4649/kube-rbac-proxy-main/0.log" Aug 04 10:07:28.920231 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.920177 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-j78v6_019cf2ad-7780-423b-a94f-2fd9badc4649/kube-rbac-proxy-self/0.log" Aug 04 10:07:28.997877 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:28.997847 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-4zbwb_8730d5cb-21cf-43fe-b552-4aecc6feb94d/monitoring-plugin/0.log" Aug 04 10:07:29.136374 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.136300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrqkf_c61bb269-a513-45f4-9926-aa19d9fd246d/node-exporter/0.log" Aug 04 10:07:29.176335 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.176305 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrqkf_c61bb269-a513-45f4-9926-aa19d9fd246d/kube-rbac-proxy/0.log" Aug 04 10:07:29.217496 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.217404 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrqkf_c61bb269-a513-45f4-9926-aa19d9fd246d/init-textfile/0.log" Aug 04 10:07:29.364027 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.363976 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-tgftq_c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52/kube-rbac-proxy-main/0.log" Aug 04 10:07:29.391126 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.391052 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-tgftq_c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52/kube-rbac-proxy-self/0.log" Aug 04 10:07:29.414833 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.413661 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-tgftq_c4b9cd19-8aaa-458c-8d1c-ad9cd6698e52/openshift-state-metrics/0.log" Aug 04 10:07:29.719179 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.719142 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-sr8qf_b0cd79c3-d796-4f80-875c-9e08fe208f7a/prometheus-operator-admission-webhook/0.log" Aug 04 10:07:29.759975 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.759939 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-786fb7c68f-rsss9_4fe431d7-ee2c-4953-9c9f-315fd661dc8b/telemeter-client/0.log" Aug 04 10:07:29.787115 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.787078 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-786fb7c68f-rsss9_4fe431d7-ee2c-4953-9c9f-315fd661dc8b/reload/0.log" Aug 04 10:07:29.816712 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.816658 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-786fb7c68f-rsss9_4fe431d7-ee2c-4953-9c9f-315fd661dc8b/kube-rbac-proxy/0.log" Aug 04 10:07:29.862356 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.862305 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c88ff454b-bjmcj_1de61d19-02e9-492b-b2c9-80c27f8937c4/thanos-query/0.log" Aug 04 10:07:29.885620 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.885590 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c88ff454b-bjmcj_1de61d19-02e9-492b-b2c9-80c27f8937c4/kube-rbac-proxy-web/0.log" Aug 04 10:07:29.913272 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.913129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c88ff454b-bjmcj_1de61d19-02e9-492b-b2c9-80c27f8937c4/kube-rbac-proxy/0.log" Aug 04 10:07:29.941009 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.940968 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c88ff454b-bjmcj_1de61d19-02e9-492b-b2c9-80c27f8937c4/prom-label-proxy/0.log" Aug 04 10:07:29.968762 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.968733 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c88ff454b-bjmcj_1de61d19-02e9-492b-b2c9-80c27f8937c4/kube-rbac-proxy-rules/0.log" Aug 04 10:07:29.994457 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:29.994378 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c88ff454b-bjmcj_1de61d19-02e9-492b-b2c9-80c27f8937c4/kube-rbac-proxy-metrics/0.log" Aug 04 10:07:31.522282 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.522225 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb"] Aug 04 10:07:31.528709 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.528685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.537894 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.537724 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb"] Aug 04 10:07:31.663903 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.663871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-sys\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.664070 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.663935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-lib-modules\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.664070 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.663987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-proc\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.664070 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.664006 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv9x6\" (UniqueName: \"kubernetes.io/projected/3f162f28-937a-4802-be65-652215aa9f4f-kube-api-access-sv9x6\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.664070 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.664067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-podres\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.765588 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-proc\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.765796 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv9x6\" (UniqueName: \"kubernetes.io/projected/3f162f28-937a-4802-be65-652215aa9f4f-kube-api-access-sv9x6\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.765796 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-podres\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.765796 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-sys\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.765796 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765705 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-proc\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.765796 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-sys\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.766083 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-lib-modules\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.766083 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.765950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-lib-modules\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.766083 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.766031 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f162f28-937a-4802-be65-652215aa9f4f-podres\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.776527 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.776456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv9x6\" (UniqueName: \"kubernetes.io/projected/3f162f28-937a-4802-be65-652215aa9f4f-kube-api-access-sv9x6\") pod \"perf-node-gather-daemonset-wj5xb\" (UID: \"3f162f28-937a-4802-be65-652215aa9f4f\") " pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:31.843219 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:31.843183 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:32.038786 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:32.037215 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb"] Aug 04 10:07:32.039479 ip-10-0-136-20 kubenswrapper[2562]: W0804 10:07:32.039441 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f162f28_937a_4802_be65_652215aa9f4f.slice/crio-e802dba250757aea63500a307b811a941c8e78047b2faf6c22ef870545dd7a6a WatchSource:0}: Error finding container e802dba250757aea63500a307b811a941c8e78047b2faf6c22ef870545dd7a6a: Status 404 returned error can't find the container with id e802dba250757aea63500a307b811a941c8e78047b2faf6c22ef870545dd7a6a Aug 04 10:07:32.512366 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:32.512341 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74575d746f-ttgrx_594408d8-1ffe-40eb-92bf-0ff640a594fe/console/0.log" Aug 04 10:07:33.033228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:33.033189 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" event={"ID":"3f162f28-937a-4802-be65-652215aa9f4f","Type":"ContainerStarted","Data":"f6a5fbe8d7bae59030b81989a8dbf2780cafb2d2b58244a33e59a77dc8a8fafe"} Aug 04 10:07:33.033690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:33.033276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" event={"ID":"3f162f28-937a-4802-be65-652215aa9f4f","Type":"ContainerStarted","Data":"e802dba250757aea63500a307b811a941c8e78047b2faf6c22ef870545dd7a6a"} Aug 04 10:07:33.033690 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:33.033311 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:33.052792 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:33.052732 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" podStartSLOduration=2.052709752 podStartE2EDuration="2.052709752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:07:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:07:33.050073375 +0000 UTC m=+3570.182282908" watchObservedRunningTime="2026-08-04 10:07:33.052709752 +0000 UTC m=+3570.184919285" Aug 04 10:07:34.426780 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:34.426750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vg8zv_fd275c7d-a708-4177-ad02-170170c21010/dns/0.log" Aug 04 10:07:34.450152 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:34.450121 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vg8zv_fd275c7d-a708-4177-ad02-170170c21010/kube-rbac-proxy/0.log" Aug 04 10:07:34.527948 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:34.527909 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sf84q_264b1ddb-48ef-4337-87c8-b107b0fce91b/dns-node-resolver/0.log" Aug 04 10:07:35.114511 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:35.114480 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lhc2c_7f7a23ce-b3e7-465c-a7a2-0b713c2b09e7/node-ca/0.log" Aug 04 10:07:36.083247 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:36.083193 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99_5e958605-71b9-4c12-946c-a38e22ddb5c6/istio-proxy/0.log" Aug 04 10:07:36.310462 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:36.310436 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-xnn4z_3ebe8684-4d19-4144-96d7-a66c6fdb78e7/kube-auth-proxy/0.log" Aug 04 10:07:36.378811 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:36.378686 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ncr4r_d43c1705-8f57-412d-bd66-cf9b141ece3b/payload-pre-processing/0.log" Aug 04 10:07:37.252228 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:37.252198 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4tpsh_e4ae14d4-e5c9-43d0-89ef-22edc19a7255/serve-healthcheck-canary/0.log" Aug 04 10:07:37.801561 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:37.801514 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cv7j8_be8cc727-6540-4150-93cf-9e35055d8f4e/kube-rbac-proxy/0.log" Aug 04 10:07:37.827366 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:37.827324 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cv7j8_be8cc727-6540-4150-93cf-9e35055d8f4e/exporter/0.log" Aug 04 10:07:37.856095 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:37.856064 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cv7j8_be8cc727-6540-4150-93cf-9e35055d8f4e/extractor/0.log" Aug 04 10:07:39.049134 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:39.049106 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ddhlv/perf-node-gather-daemonset-wj5xb" Aug 04 10:07:40.928473 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:40.928441 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xqm6t_115038de-d2ba-4e4f-8e89-bb7ccdcb6bd6/server/0.log" Aug 04 10:07:41.030750 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:41.030721 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-94szl_b2f2546a-5a85-4f91-ad18-6d404200fd5c/manager/0.log" Aug 04 10:07:41.077060 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:41.077037 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-bzzrg_9182437b-86c5-4255-8e0c-fa5a01d8b05a/manager/0.log" Aug 04 10:07:41.100321 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:41.100292 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c79dd6bb-bzzrg_9182437b-86c5-4255-8e0c-fa5a01d8b05a/copy-manifests/0.log" Aug 04 10:07:42.112605 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:42.112528 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99_5e958605-71b9-4c12-946c-a38e22ddb5c6/istio-proxy/0.log" Aug 04 10:07:42.314776 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:42.314733 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-xnn4z_3ebe8684-4d19-4144-96d7-a66c6fdb78e7/kube-auth-proxy/0.log" Aug 04 10:07:42.377326 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:42.377205 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ncr4r_d43c1705-8f57-412d-bd66-cf9b141ece3b/payload-pre-processing/0.log" Aug 04 10:07:42.581954 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:42.581923 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8856bf979-6lctf_1b34a6ca-8f30-4aeb-bd63-36aad123e6a1/manager/0.log" Aug 04 10:07:42.644920 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:42.644888 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2z4nb_ba598665-37f3-4641-917d-0d71e304b3a6/openshift-lws-operator/0.log" Aug 04 10:07:43.907072 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:43.907031 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bv99_5e958605-71b9-4c12-946c-a38e22ddb5c6/istio-proxy/0.log" Aug 04 10:07:44.125481 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:44.125415 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcc56ccff-xnn4z_3ebe8684-4d19-4144-96d7-a66c6fdb78e7/kube-auth-proxy/0.log" Aug 04 10:07:44.191354 ip-10-0-136-20 kubenswrapper[2562]: I0804 10:07:44.191324 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ncr4r_d43c1705-8f57-412d-bd66-cf9b141ece3b/payload-pre-processing/0.log"