Sep 06 00:50:36.363579 ip-10-0-129-5 systemd[1]: Starting Kubernetes Kubelet...
Sep 06 00:50:36.825721 ip-10-0-129-5 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 00:50:36.825721 ip-10-0-129-5 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 06 00:50:36.825721 ip-10-0-129-5 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 00:50:36.825721 ip-10-0-129-5 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 06 00:50:36.825721 ip-10-0-129-5 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 00:50:36.827062 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.826953    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 06 00:50:36.829932 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829917    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 00:50:36.829932 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829932    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829936    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829939    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829942    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829945    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829948    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829951    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829954    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829956    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829959    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829961    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829964    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829966    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829969    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829972    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829974    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829978    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829983    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829986    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 00:50:36.829995 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829990    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829993    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829995    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.829998    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830001    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830004    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830007    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830009    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830012    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830015    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830017    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830020    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830022    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830024    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830027    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830030    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830032    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830035    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830037    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830039    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 00:50:36.830459 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830042    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830045    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830048    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830050    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830053    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830057    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830061    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830064    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830067    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830070    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830073    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830075    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830078    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830081    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830084    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830086    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830089    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830091    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830094    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 00:50:36.830981 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830096    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830099    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830102    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830104    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830107    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830111    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830114    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830117    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830121    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830123    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830126    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830128    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830131    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830133    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830136    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830138    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830141    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830144    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830146    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 00:50:36.831461 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830150    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830152    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830155    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830157    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830160    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830162    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830165    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830167    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830170    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830172    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830175    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830177    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830180    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830182    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830185    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830187    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830190    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830192    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830195    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830197    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 00:50:36.831918 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830200    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830890    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830897    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830900    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830902    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830905    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830908    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830910    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830912    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830915    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830918    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830920    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830923    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830926    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830928    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830931    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830933    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830936    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830938    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830940    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 00:50:36.832411 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830944    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830946    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830949    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830953    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830955    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830957    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830960    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830962    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830964    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830967    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830970    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830972    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830975    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830977    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830979    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830982    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830984    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830987    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830990    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830992    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 00:50:36.832894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830995    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.830997    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831001    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831003    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831007    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831010    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831012    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831015    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831017    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831020    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831022    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831025    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831028    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831032    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831036    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831038    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831041    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831044    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831046    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 00:50:36.833385 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831049    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831053    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831056    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831059    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831062    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831064    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831067    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831069    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831072    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831074    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831076    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831079    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831082    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831084    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831086    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831089    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831093    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831098    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831101    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 00:50:36.833877 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831103    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831106    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831108    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831111    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831113    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831116    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831119    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831122    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831124    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831127    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831129    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831132    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831134    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831137    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831139    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831142    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831145    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831147    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831149    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831152    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 00:50:36.834341 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831154    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831157    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831540    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831545    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831548    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831550    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831553    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831556    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831559    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831561    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831564    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831567    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831570    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831572    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831576    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831580    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831583    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831586    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831589    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831592    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 00:50:36.834829 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831595    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831598    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831600    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831603    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831606    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831608    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831611    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831613    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831615    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831618    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831620    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831622    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831625    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831627    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831630    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831632    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831634    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831637    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831640    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831642    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 00:50:36.835339 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831644    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831647    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831650    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831652    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831655    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831657    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831660    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831663    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831665    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831667    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831670    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831673    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831676    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831679    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831681    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831684    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831686    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831688    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831691    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831693    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 00:50:36.835830 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831696    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831699    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831702    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831705    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831707    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831709    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831712    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831714    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831716    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831719    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831721    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831724    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831726    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831729    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831731    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831734    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831736    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831739    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831741    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 00:50:36.836333 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831744    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831748    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831754    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831757    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831759    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831762    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831764    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831768    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831770    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831773    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831775    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831778    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831781    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831783    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831786    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831788    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831790    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831793    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831795    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 00:50:36.836792 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831798    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831800    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.831802    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831884    2575 flags.go:64] FLAG: --address="0.0.0.0"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831928    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831934    2575 flags.go:64] FLAG: --anonymous-auth="true"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831938    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831944    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831948    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831952    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831956    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831960    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831964    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831967    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831970    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831974    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831978    2575 flags.go:64] FLAG: --cgroup-root=""
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831981    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831983    2575 flags.go:64] FLAG: --client-ca-file=""
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831987    2575 flags.go:64] FLAG: --cloud-provider="external"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831990    2575 flags.go:64] FLAG: --cluster-dns="[]"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831995    2575 flags.go:64] FLAG: --cluster-domain=""
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.831998    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832001    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 06 00:50:36.837251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832004    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832007    2575 flags.go:64] FLAG: --container-log-max-files="5"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832010    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832013    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832016    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832020    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832023    2575 flags.go:64] FLAG: --contention-profiling="false"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832025    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832028    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832031    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832034    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832038    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832040    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832043    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832046    2575 flags.go:64] FLAG: --enable-load-reader="false"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832050    2575 flags.go:64] FLAG: --enable-server="true"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832053    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832057    2575 flags.go:64] FLAG: --event-burst="100"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832059    2575 flags.go:64] FLAG: --event-qps="50"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832062    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832066    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832069    2575 flags.go:64] FLAG: --eviction-hard=""
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832072    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832076    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832078    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 06 00:50:36.837823 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832081    2575 flags.go:64] FLAG: --eviction-soft=""
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832084    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832087    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832090    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832092    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832095    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832098    2575 flags.go:64] FLAG: --fail-swap-on="true"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832100    2575 flags.go:64] FLAG: --feature-gates=""
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832104    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832107    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832110    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832113    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832116    2575 flags.go:64] FLAG: --healthz-port="10248"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832119    2575 flags.go:64] FLAG: --help="false"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832121    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-129-5.ec2.internal"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832124    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832127    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832130    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832133    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832136    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832139    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832142    2575 flags.go:64] FLAG: --image-service-endpoint=""
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832146    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 06 00:50:36.838419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832148    2575 flags.go:64] FLAG: --kube-api-burst="100"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832151    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832154    2575 flags.go:64] FLAG: --kube-api-qps="50"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832157    2575 flags.go:64] FLAG: --kube-reserved=""
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832159    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832163    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832166    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832170    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832172    2575 flags.go:64] FLAG: --lock-file=""
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832175    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832178    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832181    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832186    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832189    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832191    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832194    2575 flags.go:64] FLAG: --logging-format="text"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832197    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832201    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832204    2575 flags.go:64] FLAG: --manifest-url=""
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832207    2575 flags.go:64] FLAG: --manifest-url-header=""
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832211    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832214    2575 flags.go:64] FLAG: --max-open-files="1000000"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832217    2575 flags.go:64] FLAG: --max-pods="110"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832220    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832223    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 06 00:50:36.838990 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832226    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832228    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832231    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832234    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832237    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832243    2575 flags.go:64] FLAG: --node-status-max-images="50"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832248    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832250    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832253    2575 flags.go:64] FLAG: --pod-cidr=""
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832256    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832261    2575 flags.go:64] FLAG: --pod-manifest-path=""
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832264    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832267    2575 flags.go:64] FLAG: --pods-per-core="0"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832270    2575 flags.go:64] FLAG: --port="10250"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832273    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832291    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c16db80a0d194d7d"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832294    2575 flags.go:64] FLAG: --qos-reserved=""
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832297    2575 flags.go:64] FLAG: --read-only-port="10255"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832300    2575 flags.go:64] FLAG: --register-node="true"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832303    2575 flags.go:64] FLAG: --register-with-taints=""
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832307    2575 flags.go:64] FLAG: --registry-burst="10"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832310    2575 flags.go:64] FLAG: --registry-qps="5"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832313    2575 flags.go:64] FLAG: --reserved-cpus=""
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832315    2575 flags.go:64] FLAG: --reserved-memory=""
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832320    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 06 00:50:36.839620 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832323    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832325    2575 flags.go:64] FLAG: --rotate-certificates="false"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832328    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832331    2575 flags.go:64] FLAG: --runonce="false"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832334    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832337    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832340    2575 flags.go:64] FLAG: --seccomp-default="false"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832343    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832345    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832348    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832351    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832354    2575 flags.go:64] FLAG: --storage-driver-password="root"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832357    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832362    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832364    2575 flags.go:64] FLAG: --storage-driver-user="root"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832368    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832370    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832373    2575 flags.go:64] FLAG: --system-cgroups=""
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832376    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832381    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832384    2575 flags.go:64] FLAG: --tls-cert-file=""
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832388    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832393    2575 flags.go:64] FLAG: --tls-min-version=""
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832396    2575 flags.go:64] FLAG: --tls-private-key-file=""
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832398    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 06 00:50:36.840214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832401    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832404    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832407    2575 flags.go:64] FLAG: --v="2"
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832411    2575 flags.go:64] FLAG: --version="false"
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832420    2575 flags.go:64] FLAG: --vmodule=""
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832424    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.832427    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832544    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832549    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832552    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832555    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832558    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832561    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832564    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832566    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832569    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832571    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832574    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832576    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832579    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832583    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 00:50:36.840847 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832585    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832588    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832590    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832593    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832595    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832598    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832601    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832603    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832607    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832609    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832612    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832615    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832617    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832620    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832622    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832625    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832627    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832630    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832632    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832635    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 00:50:36.841405 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832637    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832640    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832642    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832644    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832647    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832650    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832652    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832655    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832657    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832659    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832662    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832666    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832669    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832673    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832676    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832678    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832681    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832683    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832686    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832688    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 00:50:36.841888 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832691    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832695    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832698    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832701    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832703    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832706    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832708    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832711    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832713    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832715    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832718    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832721    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832723    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832726    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832728    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832732    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832736    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832739    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832742    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832745    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 00:50:36.842380 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832748    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832750    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832753    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832756    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832759    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832761    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832764    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832766    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832770    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832774    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832776    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832779    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832782    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832784    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832787    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832790    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832792    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832795    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832797    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 00:50:36.842857 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832800    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832802    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832805    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832807    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832810    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.832812    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.833402    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.840351    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.840366    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.841733    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.841742    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841797    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841802    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841806    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841809    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 00:50:36.843535 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841812    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841815    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841818    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841821    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841824    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841826    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841829    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841831    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841834    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841836    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841839    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841842    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841844    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841847    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841849    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841852    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841855    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841857    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841859    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841862    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 00:50:36.843912 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841865    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841868    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841870    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841872    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841875    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841878    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841881    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841883    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841886    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841888    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841890    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841893    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841896    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841898    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841901    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841903    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841906    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841908    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841910    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 00:50:36.844422 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841914    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841918    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841920    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841923    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841926    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841928    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841931    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841933    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841935    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841938    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841940    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841943    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841945    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841948    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841950    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841952    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841955    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841957    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841960    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841963    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 00:50:36.844894 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841966    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841968    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841971    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841973    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841975    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841977    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841980    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841982    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841985    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841987    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841990    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841992    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841994    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841996    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.841999    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842001    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842003    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842006    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842008    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842010    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 00:50:36.845390 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842013    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842015    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842017    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842021    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842025    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842028    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842031    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842034    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842037    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842039    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842042    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842044    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842047    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842049    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842052    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 00:50:36.845873 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842055    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.842060    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842169    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842173    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842176    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842179    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842181    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842184    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842186    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842188    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842191    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842193    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842195    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842198    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842200    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 00:50:36.846245 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842203    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842205    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842207    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842209    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842212    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842214    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842217    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842219    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842221    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842224    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842226    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842229    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842231    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842234    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842236    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842239    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842241    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842244    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842246    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842249    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 00:50:36.846615 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842251    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842254    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842258    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842262    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842265    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842268    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842271    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842273    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842276    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842278    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842293    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842295    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842298    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842300    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842302    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842305    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842307    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842310    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842313    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 00:50:36.847088 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842316    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842318    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842321    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842323    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842326    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842328    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842331    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842333    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842337    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842341    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842344    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842347    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842350    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842352    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842355    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842358    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842360    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842363    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842365    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842367    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 00:50:36.847676 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842370    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842372    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842374    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842377    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842379    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842382    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842384    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842387    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842389    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842392    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842395    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842397    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842400    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842402    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842405    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842407    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842410    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842412    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842415    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842417    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 00:50:36.848152 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842420    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842422    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842425    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842428    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842430    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842433    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:36.842435    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.842439    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.842539    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.844437    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.847377    2575 server.go:1019] "Starting client certificate rotation"
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.847475    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 00:50:36.848648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.848344    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 00:50:36.851552 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.851377    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 00:50:36.854316 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.854295    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 00:50:36.863894 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.863793    2575 log.go:25] "Validated CRI v1 runtime API"
Sep 06 00:50:36.866800 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.866786    2575 log.go:25] "Validated CRI v1 image API"
Sep 06 00:50:36.867614 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.867598    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 06 00:50:36.871222 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.871204    2575 fs.go:135] Filesystem UUIDs: map[4da13541-24f6-4ad4-87fa-aa7d28a48b08:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f026eb8e-1ff2-4d4a-9f87-6f097dcb4fd2:/dev/nvme0n1p3]
Sep 06 00:50:36.871305 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.871221    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 06 00:50:36.876807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.876700    2575 manager.go:216] Machine: {Timestamp:2026-09-06 00:50:36.874790428 +0000 UTC m=+0.386718962 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116475 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec267aee0fa12b6ae417f399af9634c7 SystemUUID:ec267aee-0fa1-2b6a-e417-f399af9634c7 BootID:60fe484d-48c9-478b-98f2-d9286ffa406d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:30:dc:e1:ad:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:30:dc:e1:ad:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:9a:4c:3a:7d:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 06 00:50:36.876807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.876805    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 06 00:50:36.876914 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.876884    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 06 00:50:36.879940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.879914    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 06 00:50:36.880071 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.879942    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-5.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 06 00:50:36.880792 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.880777    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 00:50:36.881082 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.881072    2575 topology_manager.go:138] "Creating topology manager with none policy"
Sep 06 00:50:36.881121 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.881085    2575 container_manager_linux.go:309] "Creating device plugin manager"
Sep 06 00:50:36.881121 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.881093    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 00:50:36.881856 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.881845    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 00:50:36.883684 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.883673    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 06 00:50:36.887377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887366    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 00:50:36.887490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887480    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 06 00:50:36.887521 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887511    2575 kubelet.go:486] "Attempting to sync node with API server"
Sep 06 00:50:36.887521 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887521    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 06 00:50:36.887577 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887532    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 06 00:50:36.887577 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887542    2575 kubelet.go:398] "Adding apiserver pod source"
Sep 06 00:50:36.887577 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887550    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 06 00:50:36.887882 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887869    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 00:50:36.887882 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.887887    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 00:50:36.890965 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.890947    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 06 00:50:36.894219 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894201    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 06 00:50:36.894316 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894233    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 06 00:50:36.894405 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894394    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 06 00:50:36.894443 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894409    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 06 00:50:36.894443 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894415    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 06 00:50:36.894443 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894421    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 06 00:50:36.894443 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894429    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 06 00:50:36.894443 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.894435    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 06 00:50:36.895039 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895028    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 06 00:50:36.895071 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895041    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 06 00:50:36.895071 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895048    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 06 00:50:36.895071 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895053    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 06 00:50:36.895071 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895062    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 06 00:50:36.895071 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895070    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 06 00:50:36.895747 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895724    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 06 00:50:36.895747 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.895750    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 06 00:50:36.898571 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.898553    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-5.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 06 00:50:36.898988 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.898958    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 06 00:50:36.898988 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.898961    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 06 00:50:36.899771 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.899758    2575 server.go:1268] "Started kubelet"
Sep 06 00:50:36.899871 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.899841    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 06 00:50:36.899871 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.899853    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 06 00:50:36.899985 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.899893    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 06 00:50:36.900621 ip-10-0-129-5 systemd[1]: Started Kubernetes Kubelet.
Sep 06 00:50:36.900965 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.900945    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 06 00:50:36.903051 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.903034    2575 server.go:310] "Adding debug handlers to kubelet server"
Sep 06 00:50:36.905037 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.905004    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 06 00:50:36.906075 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906057    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 06 00:50:36.906167 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906074    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 06 00:50:36.906167 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.906161    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:36.906941 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906925    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 06 00:50:36.906941 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906941    2575 factory.go:55] Registering systemd factory
Sep 06 00:50:36.907089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906948    2575 factory.go:223] Registration of the systemd container factory successfully
Sep 06 00:50:36.907089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906949    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 06 00:50:36.907089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.906964    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 06 00:50:36.907089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907043    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 06 00:50:36.907089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907058    2575 reconstruct.go:97] "Volume reconstruction finished"
Sep 06 00:50:36.907089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907064    2575 reconciler.go:29] "Reconciler: start to sync state"
Sep 06 00:50:36.907393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907185    2575 factory.go:153] Registering CRI-O factory
Sep 06 00:50:36.907393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907195    2575 factory.go:223] Registration of the crio container factory successfully
Sep 06 00:50:36.907393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907210    2575 factory.go:103] Registering Raw factory
Sep 06 00:50:36.907393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907225    2575 manager.go:1196] Started watching for new ooms in manager
Sep 06 00:50:36.907587 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.907573    2575 manager.go:349] Starting recovery of all containers
Sep 06 00:50:36.908395 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.907454    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-5.ec2.internal.18d29501c7d665d0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-5.ec2.internal,UID:ip-10-0-129-5.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-5.ec2.internal,},FirstTimestamp:2026-09-06 00:50:36.899730896 +0000 UTC m=+0.411659418,LastTimestamp:2026-09-06 00:50:36.899730896 +0000 UTC m=+0.411659418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-5.ec2.internal,}"
Sep 06 00:50:36.915515 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.915493    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h5kjg"
Sep 06 00:50:36.917099 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.917070    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 06 00:50:36.917209 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.917183    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 06 00:50:36.917814 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.917795    2575 manager.go:354] Recovery completed
Sep 06 00:50:36.921441 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.921425    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h5kjg"
Sep 06 00:50:36.923505 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.923493    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:36.926147 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926131    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:36.926147 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926157    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:36.926257 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926167    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:36.926718 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926702    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 06 00:50:36.926718 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926716    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:36.926846 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926717    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 06 00:50:36.926846 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.926760    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 00:50:36.927644 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.927628    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:36.927691 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.927655    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:36.927691 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.927665    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:36.928857 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.928844    2575 policy_none.go:49] "None policy: Start"
Sep 06 00:50:36.928934 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.928862    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 06 00:50:36.928934 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.928878    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 06 00:50:36.930883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.930870    2575 policy_none.go:47] "Start"
Sep 06 00:50:36.969858 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.969843    2575 manager.go:337] "Starting Device Plugin manager"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.969885    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.969895    2575 server.go:87] "Starting device plugin registration server"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.970137    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.970154    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.970244    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.970342    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:36.970350    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.970796    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 06 00:50:36.988232 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:36.970841    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.011317 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.011272    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 06 00:50:37.012593 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.012577    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 06 00:50:37.012653 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.012625    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 06 00:50:37.012653 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.012648    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 06 00:50:37.012751 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.012684    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 06 00:50:37.014505 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.014491    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:37.070544 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.070518    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:37.071414 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.071397    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:37.071491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.071435    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:37.071491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.071448    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:37.071491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.071471    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.079603 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.079542    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.079603 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.079564    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-5.ec2.internal\": node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.093239 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.093215    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.113469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.113437    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"]
Sep 06 00:50:37.113537 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.113504    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:37.114246 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.114222    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:37.114345 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.114255    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:37.114345 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.114265    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:37.116327 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116315    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:37.116432 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116416    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.116468 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116448    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:37.116984 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116963    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:37.116984 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116970    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:37.117072 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116990    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:37.117072 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.117000    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:37.117072 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.116991    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:37.117160 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.117078    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:37.118989 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.118974    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.119063 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.118999    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 00:50:37.119603 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.119588    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 00:50:37.119688 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.119611    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 00:50:37.119688 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.119624    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeHasSufficientPID"
Sep 06 00:50:37.141917 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.141892    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-5.ec2.internal\" not found" node="ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.146147 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.146131    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-5.ec2.internal\" not found" node="ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.194233 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.194203    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.208589 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.208565    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/294d38c9919df9b4e3fbef92a3c28bbf-config\") pod \"kube-apiserver-proxy-ip-10-0-129-5.ec2.internal\" (UID: \"294d38c9919df9b4e3fbef92a3c28bbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.208692 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.208591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0566a628069d9b7f51f07b0b2331a83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal\" (UID: \"b0566a628069d9b7f51f07b0b2331a83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.208692 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.208608    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0566a628069d9b7f51f07b0b2331a83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal\" (UID: \"b0566a628069d9b7f51f07b0b2331a83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.295001 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.294978    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.309361 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.309338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/294d38c9919df9b4e3fbef92a3c28bbf-config\") pod \"kube-apiserver-proxy-ip-10-0-129-5.ec2.internal\" (UID: \"294d38c9919df9b4e3fbef92a3c28bbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.309434 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.309368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0566a628069d9b7f51f07b0b2331a83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal\" (UID: \"b0566a628069d9b7f51f07b0b2331a83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.309434 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.309384    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0566a628069d9b7f51f07b0b2331a83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal\" (UID: \"b0566a628069d9b7f51f07b0b2331a83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.309434 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.309431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0566a628069d9b7f51f07b0b2331a83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal\" (UID: \"b0566a628069d9b7f51f07b0b2331a83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.309559 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.309438    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/294d38c9919df9b4e3fbef92a3c28bbf-config\") pod \"kube-apiserver-proxy-ip-10-0-129-5.ec2.internal\" (UID: \"294d38c9919df9b4e3fbef92a3c28bbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.309559 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.309496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0566a628069d9b7f51f07b0b2331a83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal\" (UID: \"b0566a628069d9b7f51f07b0b2331a83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.395772 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.395705    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.447271 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.447240    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.449366 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.449351    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.495919 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.495895    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.596448 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.596421    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.696999 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.696934    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-5.ec2.internal\" not found"
Sep 06 00:50:37.775398 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.775373    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:37.781110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.781093    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:37.807140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.807117    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.818908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.818888    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 00:50:37.819659 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.819647    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal"
Sep 06 00:50:37.830376 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.830358    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 00:50:37.845515 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.845495    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 06 00:50:37.845621 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.845600    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 00:50:37.845671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.845600    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 00:50:37.845671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.845606    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 00:50:37.887720 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.887693    2575 apiserver.go:52] "Watching apiserver"
Sep 06 00:50:37.902181 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.902158    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 06 00:50:37.902523 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.902503    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal","openshift-network-diagnostics/network-check-target-cd6xc","openshift-network-operator/iptables-alerter-rqmff","kube-system/global-pull-secret-syncer-xsbhl","kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mw644","openshift-multus/multus-additional-cni-plugins-7txcq","openshift-multus/multus-c4mkc","openshift-multus/network-metrics-daemon-g2nfb","openshift-ovn-kubernetes/ovnkube-node-qmkb4","kube-system/konnectivity-agent-2csv8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj","openshift-dns/node-resolver-pjw4s","openshift-image-registry/node-ca-b9r8m"]
Sep 06 00:50:37.904940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.904921    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.906418 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.906398    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 06 00:50:37.906961 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.906942    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.907045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.907008    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:37.907114 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.907062    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 06 00:50:37.907204 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.907176    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.907421 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.907400    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 06 00:50:37.907503 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.907464    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 06 00:50:37.907560 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.907496    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p9qps\""
Sep 06 00:50:37.909261 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.909141    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.909261 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.909152    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.909261 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.909141    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n7hjk\""
Sep 06 00:50:37.909261 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.909223    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:37.911107 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.911087    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 06 00:50:37.911339 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.911321    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.911474 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.911454    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.911571 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.911460    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:37.911571 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.911543    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:37.911571 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.911552    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cj6x7\""
Sep 06 00:50:37.912585 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912567    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-system-cni-dir\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.912658 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912594    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cnibin\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.912658 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4141a1cc-d62d-4397-9b44-097d6e0f3229-tmp-dir\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:37.912658 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a12ab88b-8003-4b2d-b361-290028a4a1a0-host-slash\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:37.912807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912672    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnb8\" (UniqueName: \"kubernetes.io/projected/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-kube-api-access-qfnb8\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.912807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912715    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlcxf\" (UniqueName: \"kubernetes.io/projected/4141a1cc-d62d-4397-9b44-097d6e0f3229-kube-api-access-mlcxf\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:37.912807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912755    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a12ab88b-8003-4b2d-b361-290028a4a1a0-iptables-alerter-script\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:37.912807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.912807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912804    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.913014 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912829    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.913014 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4141a1cc-d62d-4397-9b44-097d6e0f3229-hosts-file\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:37.913014 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cni-binary-copy\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:37.913014 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.912905    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d88gr\" (UniqueName: \"kubernetes.io/projected/a12ab88b-8003-4b2d-b361-290028a4a1a0-kube-api-access-d88gr\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:37.913453 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.913439    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:37.915379 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.915361    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.915501 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.915393    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vgddn\""
Sep 06 00:50:37.915501 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.915425    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.915651 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.915637    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:37.915727 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.915712    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:37.917894 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.917876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:37.918150 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.918132    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 00:50:37.919812 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.919784    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 06 00:50:37.919812 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.919788    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8lbrd\""
Sep 06 00:50:37.920057 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.920041    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:37.920128 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:37.920096    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:37.920186 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.920154    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.920236 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.920191    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.922341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.922324    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:37.924364 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924339    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 00:45:38 +0000 UTC" deadline="2028-04-05 16:19:52.614030971 +0000 UTC"
Sep 06 00:50:37.924364 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924357    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ggz6l\""
Sep 06 00:50:37.924469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924365    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13863h29m14.689669426s"
Sep 06 00:50:37.924469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924372    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 06 00:50:37.924469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924383    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.924469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924406    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 06 00:50:37.924606 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924488    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 06 00:50:37.924702 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.924686    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:37.925042 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.925019    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.925218 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.925189    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 06 00:50:37.927954 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.927487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:37.928090 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.928072    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-h8fgp\""
Sep 06 00:50:37.928490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.928469    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 06 00:50:37.928566 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.928481    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 06 00:50:37.929420 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.929389    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 06 00:50:37.929672 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.929658    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 06 00:50:37.929739 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.929726    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l6c2f\""
Sep 06 00:50:37.929796 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.929781    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 06 00:50:37.930578 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.930560    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:37.932420 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.932404    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 06 00:50:37.932508 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.932429    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fq8l4\""
Sep 06 00:50:37.945628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.945611    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-89nz9"
Sep 06 00:50:37.953771 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:37.953717    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-89nz9"
Sep 06 00:50:38.008633 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.008609    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 06 00:50:38.013524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/551f2c61-9016-4ba6-94bf-ac19032e503e-cni-binary-copy\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.013619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c5308096-acd0-49da-9d37-ef559fd0e363-dbus\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:38.013619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013557    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-serviceca\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:38.013619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013571    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-tuned\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.013619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013588    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cni-binary-copy\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.013619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013612    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-socket-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c5308096-acd0-49da-9d37-ef559fd0e363-kubelet-config\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysctl-conf\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cnibin\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013706    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysctl-d\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013758    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-tmp\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cnibin\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-etc-selinux\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-etc-kubernetes\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.013869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013862    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-var-lib-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-conf-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013913    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdtp2\" (UniqueName: \"kubernetes.io/projected/e887d0d0-952c-470f-bf00-4664435b1043-kube-api-access-bdtp2\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d88gr\" (UniqueName: \"kubernetes.io/projected/a12ab88b-8003-4b2d-b361-290028a4a1a0-kube-api-access-d88gr\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.013975    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014000    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-kubelet\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4141a1cc-d62d-4397-9b44-097d6e0f3229-tmp-dir\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-node-log\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014092    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-env-overrides\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ad2e909-80a7-454e-ae32-86257fc0c2db-agent-certs\") pod \"konnectivity-agent-2csv8\" (UID: \"3ad2e909-80a7-454e-ae32-86257fc0c2db\") " pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014147    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-daemon-config\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014172    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014154    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cni-binary-copy\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-kubernetes\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-ovn\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlcxf\" (UniqueName: \"kubernetes.io/projected/4141a1cc-d62d-4397-9b44-097d6e0f3229-kube-api-access-mlcxf\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:38.014341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014345    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a12ab88b-8003-4b2d-b361-290028a4a1a0-iptables-alerter-script\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014360    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4141a1cc-d62d-4397-9b44-097d6e0f3229-tmp-dir\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-kubelet\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsq4w\" (UniqueName: \"kubernetes.io/projected/e3cde646-3f9b-4bd3-b936-7f3579602519-kube-api-access-lsq4w\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-k8s-cni-cncf-io\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-hostroot\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014787    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-log-socket\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-device-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014843    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a12ab88b-8003-4b2d-b361-290028a4a1a0-iptables-alerter-script\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv5xp\" (UniqueName: \"kubernetes.io/projected/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-kube-api-access-gv5xp\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014876    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-run-netns\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-cni-bin\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-ovnkube-config\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-system-cni-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014974    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-socket-dir-parent\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.014992    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-host\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:38.015209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015030    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-systemd\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-sys\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-system-cni-dir\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a12ab88b-8003-4b2d-b361-290028a4a1a0-host-slash\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015120    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-systemd-units\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-system-cni-dir\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-multus-certs\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a12ab88b-8003-4b2d-b361-290028a4a1a0-host-slash\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-slash\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rskfz\" (UniqueName: \"kubernetes.io/projected/239122a3-2aee-4587-b6ea-694cf69928cd-kube-api-access-rskfz\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-cni-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fkq\" (UniqueName: \"kubernetes.io/projected/551f2c61-9016-4ba6-94bf-ac19032e503e-kube-api-access-p2fkq\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015308    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4141a1cc-d62d-4397-9b44-097d6e0f3229-hosts-file\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-etc-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015450    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4141a1cc-d62d-4397-9b44-097d6e0f3229-hosts-file\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:38.016018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015678    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-cni-netd\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-ovnkube-script-lib\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysconfig\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-run\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015815    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ad2e909-80a7-454e-ae32-86257fc0c2db-konnectivity-ca\") pod \"konnectivity-agent-2csv8\" (UID: \"3ad2e909-80a7-454e-ae32-86257fc0c2db\") " pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-netns\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-cni-bin\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-host\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015915    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b575k\" (UniqueName: \"kubernetes.io/projected/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-kube-api-access-b575k\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-cnibin\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015940    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-cni-multus\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-modprobe-d\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.015998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-lib-modules\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnb8\" (UniqueName: \"kubernetes.io/projected/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-kube-api-access-qfnb8\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.016940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-systemd\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016080    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016103    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/239122a3-2aee-4587-b6ea-694cf69928cd-ovn-node-metrics-cert\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016144    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-var-lib-kubelet\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016162    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016203    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-registration-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016227    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-sys-fs\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016322    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.017573 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.016589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.020454 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.020436    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 06 00:50:38.024026 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.024003    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlcxf\" (UniqueName: \"kubernetes.io/projected/4141a1cc-d62d-4397-9b44-097d6e0f3229-kube-api-access-mlcxf\") pod \"node-resolver-pjw4s\" (UID: \"4141a1cc-d62d-4397-9b44-097d6e0f3229\") " pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:38.024097 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.024084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnb8\" (UniqueName: \"kubernetes.io/projected/6b7afb0c-24a3-4f9f-a295-e1d8c426e03c-kube-api-access-qfnb8\") pod \"multus-additional-cni-plugins-7txcq\" (UID: \"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c\") " pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:38.024133 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.024111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d88gr\" (UniqueName: \"kubernetes.io/projected/a12ab88b-8003-4b2d-b361-290028a4a1a0-kube-api-access-d88gr\") pod \"iptables-alerter-rqmff\" (UID: \"a12ab88b-8003-4b2d-b361-290028a4a1a0\") " pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:38.078343 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:38.078312    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294d38c9919df9b4e3fbef92a3c28bbf.slice/crio-ace9332d57be4984eee01fda61f2180f0fcb1fe174b364cc717ae8a9d9f9b36f WatchSource:0}: Error finding container ace9332d57be4984eee01fda61f2180f0fcb1fe174b364cc717ae8a9d9f9b36f: Status 404 returned error can't find the container with id ace9332d57be4984eee01fda61f2180f0fcb1fe174b364cc717ae8a9d9f9b36f
Sep 06 00:50:38.079240 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:38.079216    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0566a628069d9b7f51f07b0b2331a83.slice/crio-b449de2e8301fe88a9a836e501350f27c893d87b929d9a038cddc6f6203467d0 WatchSource:0}: Error finding container b449de2e8301fe88a9a836e501350f27c893d87b929d9a038cddc6f6203467d0: Status 404 returned error can't find the container with id b449de2e8301fe88a9a836e501350f27c893d87b929d9a038cddc6f6203467d0
Sep 06 00:50:38.083762 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:38.083745    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 00:50:39.299202 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-etc-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299272 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299272 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-cni-netd\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299272 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299240    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-ovnkube-script-lib\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299272 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299254    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysconfig\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299287    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-etc-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-run\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299306    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-cni-netd\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299343    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299383    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ad2e909-80a7-454e-ae32-86257fc0c2db-konnectivity-ca\") pod \"konnectivity-agent-2csv8\" (UID: \"3ad2e909-80a7-454e-ae32-86257fc0c2db\") " pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysconfig\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299408    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-netns\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-run\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299436    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-netns\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-cni-bin\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299494    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-host\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299513    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-cni-bin\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299529    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b575k\" (UniqueName: \"kubernetes.io/projected/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-kube-api-access-b575k\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-cnibin\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-host\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-cni-multus\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-cnibin\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299628    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-modprobe-d\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299650    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-cni-multus\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-lib-modules\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-systemd\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299714    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299720    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-modprobe-d\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/239122a3-2aee-4587-b6ea-694cf69928cd-ovn-node-metrics-cert\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-systemd\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.299868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-var-lib-kubelet\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299773    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-ovnkube-script-lib\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299783    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-lib-modules\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299803    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-registration-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299814    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-var-lib-kubelet\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-sys-fs\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/551f2c61-9016-4ba6-94bf-ac19032e503e-cni-binary-copy\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-registration-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299902    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-sys-fs\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ad2e909-80a7-454e-ae32-86257fc0c2db-konnectivity-ca\") pod \"konnectivity-agent-2csv8\" (UID: \"3ad2e909-80a7-454e-ae32-86257fc0c2db\") " pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c5308096-acd0-49da-9d37-ef559fd0e363-dbus\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-serviceca\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-tuned\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-socket-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.299996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c5308096-acd0-49da-9d37-ef559fd0e363-dbus\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300008    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c5308096-acd0-49da-9d37-ef559fd0e363-kubelet-config\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysctl-conf\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.300798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c5308096-acd0-49da-9d37-ef559fd0e363-kubelet-config\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300073    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysctl-d\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-socket-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300093    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-tmp\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300116    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-etc-selinux\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-etc-kubernetes\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.300154    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300164    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-var-lib-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-conf-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300195    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysctl-d\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdtp2\" (UniqueName: \"kubernetes.io/projected/e887d0d0-952c-470f-bf00-4664435b1043-kube-api-access-bdtp2\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-etc-selinux\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-sysctl-conf\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300270    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-etc-kubernetes\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-kubelet\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300306    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-node-log\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.301595 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300313    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-var-lib-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-var-lib-kubelet\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-env-overrides\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300360    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-node-log\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300307    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-conf-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-serviceca\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ad2e909-80a7-454e-ae32-86257fc0c2db-agent-certs\") pod \"konnectivity-agent-2csv8\" (UID: \"3ad2e909-80a7-454e-ae32-86257fc0c2db\") " pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300408    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/551f2c61-9016-4ba6-94bf-ac19032e503e-cni-binary-copy\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-daemon-config\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-kubernetes\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.300479    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:39.800457786 +0000 UTC m=+2.129816605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-ovn\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-kubernetes\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-kubelet\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.300572    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:39.302437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.300614    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:39.800599235 +0000 UTC m=+2.129958071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-openvswitch\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-run-ovn\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsq4w\" (UniqueName: \"kubernetes.io/projected/e3cde646-3f9b-4bd3-b936-7f3579602519-kube-api-access-lsq4w\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-k8s-cni-cncf-io\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-hostroot\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-log-socket\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300785    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-k8s-cni-cncf-io\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-device-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5xp\" (UniqueName: \"kubernetes.io/projected/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-kube-api-access-gv5xp\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e3cde646-3f9b-4bd3-b936-7f3579602519-device-dir\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300829    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-daemon-config\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-run-netns\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-hostroot\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300886    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-log-socket\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300893    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-cni-bin\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-run-netns\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300926    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-cni-bin\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.300997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-ovnkube-config\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-system-cni-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301056    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-socket-dir-parent\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-host\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-system-cni-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-systemd\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-socket-dir-parent\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-sys\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-systemd\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301164    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-systemd-units\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301174    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-host\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-multus-certs\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-kubelet\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-systemd-units\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301192    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-sys\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-slash\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.303516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301221    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-host-run-multus-certs\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rskfz\" (UniqueName: \"kubernetes.io/projected/239122a3-2aee-4587-b6ea-694cf69928cd-kube-api-access-rskfz\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/239122a3-2aee-4587-b6ea-694cf69928cd-host-slash\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301383    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-cni-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301409    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-ovnkube-config\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301408    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fkq\" (UniqueName: \"kubernetes.io/projected/551f2c61-9016-4ba6-94bf-ac19032e503e-kube-api-access-p2fkq\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301642    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/551f2c61-9016-4ba6-94bf-ac19032e503e-multus-cni-dir\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.301750    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/239122a3-2aee-4587-b6ea-694cf69928cd-env-overrides\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.302763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-etc-tuned\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.302819    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/239122a3-2aee-4587-b6ea-694cf69928cd-ovn-node-metrics-cert\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.303417    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-tmp\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.304029 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.303558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ad2e909-80a7-454e-ae32-86257fc0c2db-agent-certs\") pod \"konnectivity-agent-2csv8\" (UID: \"3ad2e909-80a7-454e-ae32-86257fc0c2db\") " pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:39.305621 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.305608    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:50:39.305661 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.305628    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:50:39.305661 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.305639    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:39.305713 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.305699    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:39.805686379 +0000 UTC m=+2.135045193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:39.307602 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.307581    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b575k\" (UniqueName: \"kubernetes.io/projected/fb7ead95-02a4-4aa3-9431-ef07693c5bf0-kube-api-access-b575k\") pod \"tuned-mw644\" (UID: \"fb7ead95-02a4-4aa3-9431-ef07693c5bf0\") " pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.309123 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.309095    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsq4w\" (UniqueName: \"kubernetes.io/projected/e3cde646-3f9b-4bd3-b936-7f3579602519-kube-api-access-lsq4w\") pod \"aws-ebs-csi-driver-node-fv7tj\" (UID: \"e3cde646-3f9b-4bd3-b936-7f3579602519\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.309382 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.309333    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv5xp\" (UniqueName: \"kubernetes.io/projected/16c172de-2c01-4b51-a2ad-3e0b7b35ff3d-kube-api-access-gv5xp\") pod \"node-ca-b9r8m\" (UID: \"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d\") " pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.309828 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.309806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdtp2\" (UniqueName: \"kubernetes.io/projected/e887d0d0-952c-470f-bf00-4664435b1043-kube-api-access-bdtp2\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:39.310405 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.310386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rskfz\" (UniqueName: \"kubernetes.io/projected/239122a3-2aee-4587-b6ea-694cf69928cd-kube-api-access-rskfz\") pod \"ovnkube-node-qmkb4\" (UID: \"239122a3-2aee-4587-b6ea-694cf69928cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.310783 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.310768    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fkq\" (UniqueName: \"kubernetes.io/projected/551f2c61-9016-4ba6-94bf-ac19032e503e-kube-api-access-p2fkq\") pod \"multus-c4mkc\" (UID: \"551f2c61-9016-4ba6-94bf-ac19032e503e\") " pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.329393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.329373    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:39.418440 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.418380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7txcq"
Sep 06 00:50:39.422425 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.422409    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pjw4s"
Sep 06 00:50:39.428647 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.428628    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7afb0c_24a3_4f9f_a295_e1d8c426e03c.slice/crio-7a089d43d022ca9aa95398640f599e527cf4af6142c6b6d34bfbc2cfd70b3a1a WatchSource:0}: Error finding container 7a089d43d022ca9aa95398640f599e527cf4af6142c6b6d34bfbc2cfd70b3a1a: Status 404 returned error can't find the container with id 7a089d43d022ca9aa95398640f599e527cf4af6142c6b6d34bfbc2cfd70b3a1a
Sep 06 00:50:39.430686 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.430664    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4141a1cc_d62d_4397_9b44_097d6e0f3229.slice/crio-e39033c6b96d2b63c192def9bf4161bfe4bfe13cb55e73b896678bcb779b8fba WatchSource:0}: Error finding container e39033c6b96d2b63c192def9bf4161bfe4bfe13cb55e73b896678bcb779b8fba: Status 404 returned error can't find the container with id e39033c6b96d2b63c192def9bf4161bfe4bfe13cb55e73b896678bcb779b8fba
Sep 06 00:50:39.439725 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.439704    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rqmff"
Sep 06 00:50:39.449046 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.449019    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12ab88b_8003_4b2d_b361_290028a4a1a0.slice/crio-47754b870a2fa44cacafb8d48a46eb2046a03d547ffc1bfac23208d3b93513e7 WatchSource:0}: Error finding container 47754b870a2fa44cacafb8d48a46eb2046a03d547ffc1bfac23208d3b93513e7: Status 404 returned error can't find the container with id 47754b870a2fa44cacafb8d48a46eb2046a03d547ffc1bfac23208d3b93513e7
Sep 06 00:50:39.455023 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.455006    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mw644"
Sep 06 00:50:39.462369 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.462354    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7ead95_02a4_4aa3_9431_ef07693c5bf0.slice/crio-f6d66372e5441ab3b6e4149d2831a23c34a28633fc38e0bc7ceadb38f1992135 WatchSource:0}: Error finding container f6d66372e5441ab3b6e4149d2831a23c34a28633fc38e0bc7ceadb38f1992135: Status 404 returned error can't find the container with id f6d66372e5441ab3b6e4149d2831a23c34a28633fc38e0bc7ceadb38f1992135
Sep 06 00:50:39.478754 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.478739    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b9r8m"
Sep 06 00:50:39.486585 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.486567    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c172de_2c01_4b51_a2ad_3e0b7b35ff3d.slice/crio-7baba8f54341e78378918d9ade0438cf41664999e06a4a67d1369b8344237c07 WatchSource:0}: Error finding container 7baba8f54341e78378918d9ade0438cf41664999e06a4a67d1369b8344237c07: Status 404 returned error can't find the container with id 7baba8f54341e78378918d9ade0438cf41664999e06a4a67d1369b8344237c07
Sep 06 00:50:39.493559 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.493545    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:50:39.500195 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.500181    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:50:39.500770 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.500751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239122a3_2aee_4587_b6ea_694cf69928cd.slice/crio-62fa93a0788f6f1b078901e9f3741695e8b271d0c44c5a58150e2a1faf10578c WatchSource:0}: Error finding container 62fa93a0788f6f1b078901e9f3741695e8b271d0c44c5a58150e2a1faf10578c: Status 404 returned error can't find the container with id 62fa93a0788f6f1b078901e9f3741695e8b271d0c44c5a58150e2a1faf10578c
Sep 06 00:50:39.508364 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.508343    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad2e909_80a7_454e_ae32_86257fc0c2db.slice/crio-b53441ba91ac6afd6287ed6373497c7599015ecae97d7fa374c9535d3c6c008b WatchSource:0}: Error finding container b53441ba91ac6afd6287ed6373497c7599015ecae97d7fa374c9535d3c6c008b: Status 404 returned error can't find the container with id b53441ba91ac6afd6287ed6373497c7599015ecae97d7fa374c9535d3c6c008b
Sep 06 00:50:39.523886 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.523839    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj"
Sep 06 00:50:39.526995 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.526976    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c4mkc"
Sep 06 00:50:39.532931 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.532911    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cde646_3f9b_4bd3_b936_7f3579602519.slice/crio-4becc212912492d573e3f721c003499927b0571a4ac9b85c940cfe4588860463 WatchSource:0}: Error finding container 4becc212912492d573e3f721c003499927b0571a4ac9b85c940cfe4588860463: Status 404 returned error can't find the container with id 4becc212912492d573e3f721c003499927b0571a4ac9b85c940cfe4588860463
Sep 06 00:50:39.534844 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:50:39.534820    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551f2c61_9016_4ba6_94bf_ac19032e503e.slice/crio-3492ee68911a375f024b73e13fca436eb920c780051546bad1056e91ce2a3683 WatchSource:0}: Error finding container 3492ee68911a375f024b73e13fca436eb920c780051546bad1056e91ce2a3683: Status 404 returned error can't find the container with id 3492ee68911a375f024b73e13fca436eb920c780051546bad1056e91ce2a3683
Sep 06 00:50:39.804320 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.804233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:39.804320 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.804278    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:39.804519 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.804386    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:39.804519 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.804440    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:40.804427477 +0000 UTC m=+3.133786291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:39.804730 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.804708    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:39.804786 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.804755    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:40.804740328 +0000 UTC m=+3.134099148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:39.905353 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.905321    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:39.905503 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.905487    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:50:39.905571 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.905509    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:50:39.905571 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.905530    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:39.905654 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:39.905592    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:40.905572949 +0000 UTC m=+3.234931777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:39.932026 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:39.931996    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:40.139509 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.139265    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 00:45:39 +0000 UTC" deadline="2028-05-26 16:02:45.547100916 +0000 UTC"
Sep 06 00:50:40.139509 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.139321    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15087h12m5.407784373s"
Sep 06 00:50:40.207872 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.207798    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal" event={"ID":"294d38c9919df9b4e3fbef92a3c28bbf","Type":"ContainerStarted","Data":"ace9332d57be4984eee01fda61f2180f0fcb1fe174b364cc717ae8a9d9f9b36f"}
Sep 06 00:50:40.245341 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.245302    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c4mkc" event={"ID":"551f2c61-9016-4ba6-94bf-ac19032e503e","Type":"ContainerStarted","Data":"3492ee68911a375f024b73e13fca436eb920c780051546bad1056e91ce2a3683"}
Sep 06 00:50:40.255608 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.255540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj" event={"ID":"e3cde646-3f9b-4bd3-b936-7f3579602519","Type":"ContainerStarted","Data":"4becc212912492d573e3f721c003499927b0571a4ac9b85c940cfe4588860463"}
Sep 06 00:50:40.265348 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.265318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2csv8" event={"ID":"3ad2e909-80a7-454e-ae32-86257fc0c2db","Type":"ContainerStarted","Data":"b53441ba91ac6afd6287ed6373497c7599015ecae97d7fa374c9535d3c6c008b"}
Sep 06 00:50:40.274277 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.274249    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"62fa93a0788f6f1b078901e9f3741695e8b271d0c44c5a58150e2a1faf10578c"}
Sep 06 00:50:40.289951 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.289920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b9r8m" event={"ID":"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d","Type":"ContainerStarted","Data":"7baba8f54341e78378918d9ade0438cf41664999e06a4a67d1369b8344237c07"}
Sep 06 00:50:40.295811 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.295624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mw644" event={"ID":"fb7ead95-02a4-4aa3-9431-ef07693c5bf0","Type":"ContainerStarted","Data":"f6d66372e5441ab3b6e4149d2831a23c34a28633fc38e0bc7ceadb38f1992135"}
Sep 06 00:50:40.307556 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.307508    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pjw4s" event={"ID":"4141a1cc-d62d-4397-9b44-097d6e0f3229","Type":"ContainerStarted","Data":"e39033c6b96d2b63c192def9bf4161bfe4bfe13cb55e73b896678bcb779b8fba"}
Sep 06 00:50:40.317268 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.317214    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal" event={"ID":"b0566a628069d9b7f51f07b0b2331a83","Type":"ContainerStarted","Data":"b449de2e8301fe88a9a836e501350f27c893d87b929d9a038cddc6f6203467d0"}
Sep 06 00:50:40.339510 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.339458    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rqmff" event={"ID":"a12ab88b-8003-4b2d-b361-290028a4a1a0","Type":"ContainerStarted","Data":"47754b870a2fa44cacafb8d48a46eb2046a03d547ffc1bfac23208d3b93513e7"}
Sep 06 00:50:40.356970 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.356938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerStarted","Data":"7a089d43d022ca9aa95398640f599e527cf4af6142c6b6d34bfbc2cfd70b3a1a"}
Sep 06 00:50:40.812946 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.812346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:40.812946 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.812401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:40.812946 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.812543    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:40.812946 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.812577    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:40.812946 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.812601    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:42.812583194 +0000 UTC m=+5.141942031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:40.812946 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.812646    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:42.812627432 +0000 UTC m=+5.141986255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:40.913175 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:40.913138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:40.913355 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.913295    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:50:40.913355 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.913318    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:50:40.913355 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.913330    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:40.913508 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:40.913382    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:42.913365227 +0000 UTC m=+5.242724046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:41.006088 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.006051    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:41.140006 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.139872    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 00:45:39 +0000 UTC" deadline="2028-05-16 01:36:11.412682962 +0000 UTC"
Sep 06 00:50:41.140006 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.139955    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14832h45m30.272733046s"
Sep 06 00:50:41.195798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.195769    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:41.195985 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:41.195899    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:41.196311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.196292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:41.196404 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:41.196385    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:41.196469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.196452    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:41.196573 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:41.196552    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:41.273217 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:41.273185    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 00:50:42.828226 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:42.828188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:42.828781 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:42.828253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:42.828781 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.828400    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:42.828781 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.828471    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:46.828451442 +0000 UTC m=+9.157810263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:42.830356 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.828896    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:42.830356 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.828953    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:46.828933657 +0000 UTC m=+9.158292479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:42.928908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:42.928866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:42.929125 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.929067    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:50:42.929206 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.929136    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:50:42.929206 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.929151    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:42.929301 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:42.929210    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:46.929189067 +0000 UTC m=+9.258547887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:43.196137 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:43.196046    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:43.196299 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:43.196206    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:43.196605 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:43.196589    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:43.196693 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:43.196671    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:43.196755 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:43.196723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:43.196813 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:43.196783    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:45.195949 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:45.195904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:45.195949 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:45.195939    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:45.196409 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:45.195904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:45.196409 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:45.196043    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:45.196409 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:45.196141    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:45.196409 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:45.196219    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:46.861940 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:46.861823    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:46.862392 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:46.861996    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:46.862392 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.862041    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:46.862392 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.862094    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:46.862392 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.862106    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:54.862088745 +0000 UTC m=+17.191447576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:46.862392 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.862128    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:54.86211528 +0000 UTC m=+17.191474094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:46.962442 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:46.962408    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:46.962610 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.962582    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:50:46.962610 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.962603    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:50:46.962610 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.962612    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:46.962807 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:46.962661    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:50:54.962643166 +0000 UTC m=+17.292001981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:47.196378 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:47.196292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:47.196378 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:47.196321    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:47.196378 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:47.196337    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:47.196636 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:47.196421    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:47.196636 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:47.196517    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:47.196636 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:47.196592    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:49.196074 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:49.196040    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:49.196074 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:49.196072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:49.196696 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:49.196047    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:49.196696 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:49.196187    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:49.196696 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:49.196265    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:49.196696 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:49.196349    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:51.196048 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:51.196012    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:51.196560 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:51.196122    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:51.196560 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:51.196137    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:51.196560 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:51.196220    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:51.196560 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:51.196245    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:51.196560 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:51.196335    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:53.196059 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:53.196021    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:53.196436 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:53.196061    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:53.196436 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:53.196141    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:53.196436 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:53.196158    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:53.196436 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:53.196225    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:53.196436 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:53.196284    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:54.923680 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:54.923643    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:54.924160 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:54.923696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:54.924160 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:54.923867    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:54.924160 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:54.923874    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:54.924160 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:54.923930    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:51:10.923915659 +0000 UTC m=+33.253274473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:50:54.924160 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:54.923949    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:51:10.923939214 +0000 UTC m=+33.253298027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:50:55.025175 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:55.025135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:55.025389 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.025373    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:50:55.025444 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.025396    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:50:55.025444 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.025409    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:55.025518 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.025466    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:51:11.025448109 +0000 UTC m=+33.354806924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:50:55.196366 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:55.196282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:55.196366 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:55.196318    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:55.196366 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:55.196318    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:55.196614 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.196415    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:55.196614 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.196498    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:55.196614 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:55.196556    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:57.196438 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:57.196399    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:57.196938 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:57.196506    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:57.196938 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:57.196522    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:57.196938 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:57.196536    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:57.196938 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:57.196637    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:57.196938 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:57.196726    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:59.196194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.196035    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:50:59.196561 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.196059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:50:59.196561 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:59.196265    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:50:59.196561 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.196067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:50:59.196561 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:59.196339    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:50:59.196561 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:50:59.196427    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:50:59.438693 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.438622    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal" event={"ID":"294d38c9919df9b4e3fbef92a3c28bbf","Type":"ContainerStarted","Data":"60ae499bab9adaa602c4a3477cb37c40c7d883fab2b04a7f8c8ebad0b0121abe"}
Sep 06 00:50:59.447502 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.447381    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:50:59.449473 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.449101    2575 generic.go:356] "Generic (PLEG): container finished" podID="239122a3-2aee-4587-b6ea-694cf69928cd" containerID="9836122ce742b472dc27395e9729a488559e561b657b07873707715cb793f879" exitCode=1
Sep 06 00:50:59.449473 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.449172    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"c8cb814f82c96fd76c3f056affea454563b3b8d21759c383d53c57db117f630b"}
Sep 06 00:50:59.449473 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.449223    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerDied","Data":"9836122ce742b472dc27395e9729a488559e561b657b07873707715cb793f879"}
Sep 06 00:50:59.449473 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.449255    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"9e2aa635fe071cb077602550d7e910e3b97dbe4f440dcb498e26658f81035a82"}
Sep 06 00:50:59.451735 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.451692    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-5.ec2.internal" podStartSLOduration=21.451680053 podStartE2EDuration="21.451680053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:50:59.45162626 +0000 UTC m=+21.780985123" watchObservedRunningTime="2026-09-06 00:50:59.451680053 +0000 UTC m=+21.781038888"
Sep 06 00:50:59.461011 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.460938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mw644" event={"ID":"fb7ead95-02a4-4aa3-9431-ef07693c5bf0","Type":"ContainerStarted","Data":"d5640870b7b3a389978ec2353c95e211ab62172e5a7ef1923205393bafd45de2"}
Sep 06 00:50:59.480659 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:50:59.480599    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mw644" podStartSLOduration=2.093669119 podStartE2EDuration="21.480579982s" totalImagesPullingTime="19.386910863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:50:59.480166961 +0000 UTC m=+21.809525830" watchObservedRunningTime="2026-09-06 00:50:59.480579982 +0000 UTC m=+21.809938820"
Sep 06 00:51:00.464720 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.464433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pjw4s" event={"ID":"4141a1cc-d62d-4397-9b44-097d6e0f3229","Type":"ContainerStarted","Data":"de36617510c1f1e4a5582fc09b24d69f81c898a92a45f34d450297b4679f6266"}
Sep 06 00:51:00.466256 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.466233    2575 generic.go:356] "Generic (PLEG): container finished" podID="b0566a628069d9b7f51f07b0b2331a83" containerID="a4c6f7e349eb1b6a6c7693f0c36b6938dfb42c7f15e8a6248749163eda4bd9e1" exitCode=0
Sep 06 00:51:00.466355 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.466312    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal" event={"ID":"b0566a628069d9b7f51f07b0b2331a83","Type":"ContainerDied","Data":"a4c6f7e349eb1b6a6c7693f0c36b6938dfb42c7f15e8a6248749163eda4bd9e1"}
Sep 06 00:51:00.468228 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.468029    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rqmff" event={"ID":"a12ab88b-8003-4b2d-b361-290028a4a1a0","Type":"ContainerStarted","Data":"0a6203475e5465b08338f4338eb7c2147ed9aa2e5b7f436ab59013becda1975b"}
Sep 06 00:51:00.469809 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.469783    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b7afb0c-24a3-4f9f-a295-e1d8c426e03c" containerID="af4006c31cc982b41a590af17ae6a5ac3b3ded1cea55c9a83353033c1678c28f" exitCode=0
Sep 06 00:51:00.469948 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.469884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerDied","Data":"af4006c31cc982b41a590af17ae6a5ac3b3ded1cea55c9a83353033c1678c28f"}
Sep 06 00:51:00.471604 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.471581    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c4mkc" event={"ID":"551f2c61-9016-4ba6-94bf-ac19032e503e","Type":"ContainerStarted","Data":"801cf232c4acc2776f2beae48a51d72871db08d845e3f9a1635f2d434bc85341"}
Sep 06 00:51:00.476207 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.476184    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj" event={"ID":"e3cde646-3f9b-4bd3-b936-7f3579602519","Type":"ContainerStarted","Data":"aa7190aa952fbc581ad497e26f840ec2dfa858e22f0e85abde43e9463732d273"}
Sep 06 00:51:00.478100 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.478060    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2csv8" event={"ID":"3ad2e909-80a7-454e-ae32-86257fc0c2db","Type":"ContainerStarted","Data":"970fbc63dce00f14f06b0835f9a2a9ab498c6046e545931b086732ab2ec84216"}
Sep 06 00:51:00.481243 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.481219    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:51:00.481393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.481350    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pjw4s" podStartSLOduration=3.064625096 podStartE2EDuration="22.481334106s" totalImagesPullingTime="19.41670901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:00.480668211 +0000 UTC m=+22.810027064" watchObservedRunningTime="2026-09-06 00:51:00.481334106 +0000 UTC m=+22.810692944"
Sep 06 00:51:00.481662 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.481644    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"049fd2a9118f6cdf7efe45d6ccc0b8bd3e49362392ddf15304e0af26e2473118"}
Sep 06 00:51:00.481758 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.481714    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"c4853637a4242b8e6ac83cd8df97a4549c178b1e21c18149f90149a426b1fe3c"}
Sep 06 00:51:00.481822 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.481767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"582a884f2a3070d72d531cbb22be2fa719c912c727e4dbdfd8a37798e1ffeb15"}
Sep 06 00:51:00.483205 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.483184    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b9r8m" event={"ID":"16c172de-2c01-4b51-a2ad-3e0b7b35ff3d","Type":"ContainerStarted","Data":"ad5588a804a7dc749171e50485f2bd36456b7d70f82d22953af35eaba5d420c3"}
Sep 06 00:51:00.494216 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.494174    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rqmff" podStartSLOduration=3.094612251 podStartE2EDuration="22.494161402s" totalImagesPullingTime="19.399549151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:00.494112644 +0000 UTC m=+22.823471501" watchObservedRunningTime="2026-09-06 00:51:00.494161402 +0000 UTC m=+22.823520238"
Sep 06 00:51:00.525250 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.525200    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c4mkc" podStartSLOduration=2.648166395 podStartE2EDuration="22.525184819s" totalImagesPullingTime="19.877018424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:00.524786225 +0000 UTC m=+22.854145072" watchObservedRunningTime="2026-09-06 00:51:00.525184819 +0000 UTC m=+22.854543655"
Sep 06 00:51:00.525433 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.525416    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2csv8" podStartSLOduration=3.217371866 podStartE2EDuration="22.5254119s" totalImagesPullingTime="19.308040034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:00.5074055 +0000 UTC m=+22.836764338" watchObservedRunningTime="2026-09-06 00:51:00.5254119 +0000 UTC m=+22.854770738"
Sep 06 00:51:00.575904 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.575848    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b9r8m" podStartSLOduration=3.382812685 podStartE2EDuration="22.575835397s" totalImagesPullingTime="19.193022712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:00.575475441 +0000 UTC m=+22.904834277" watchObservedRunningTime="2026-09-06 00:51:00.575835397 +0000 UTC m=+22.905194258"
Sep 06 00:51:00.612361 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.612272    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:00.689806 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.689767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:00.883052 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:00.883030    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 06 00:51:01.018401 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.018328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:01.121299 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.121256    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:01.164976 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.164835    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T00:51:00.883048185Z","UUID":"d4b7dd80-66b6-4eca-baef-81600293e363","Handler":null,"Name":"","Endpoint":""}
Sep 06 00:51:01.167353 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.167330    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 06 00:51:01.167888 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.167366    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 06 00:51:01.196367 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.196307    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:01.196524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.196410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:01.196524 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:01.196404    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:01.196633 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:01.196520    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:01.196633 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.196565    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:01.196737 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:01.196667    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:01.424145 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.423874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:01.488657 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.488606    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj" event={"ID":"e3cde646-3f9b-4bd3-b936-7f3579602519","Type":"ContainerStarted","Data":"0d9cfa1a6ee3fc5536f8e9584d62aeb11dbd6ab9730c093e927b65cb7f5f56a9"}
Sep 06 00:51:01.491376 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.491338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal" event={"ID":"b0566a628069d9b7f51f07b0b2331a83","Type":"ContainerStarted","Data":"443b6f695e541a471a0938aa1f6bc354d5cc7a3b9c1c5dd3643d9701f55d7c73"}
Sep 06 00:51:01.541644 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:01.541584    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:02.496445 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.496407    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj" event={"ID":"e3cde646-3f9b-4bd3-b936-7f3579602519","Type":"ContainerStarted","Data":"e44902592b04254432f1e1b413413294362343537527a9096f7c02e9de94c1aa"}
Sep 06 00:51:02.499991 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.499972    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:51:02.500446 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.500424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"f05338f20f67cee3e3142f4126e4f9294bc985ed24d25c465ebb1841261d2afc"}
Sep 06 00:51:02.513995 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.513950    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fv7tj" podStartSLOduration=2.65054977 podStartE2EDuration="24.513931958s" totalImagesPullingTime="21.863382188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:02.513639909 +0000 UTC m=+24.842998962" watchObservedRunningTime="2026-09-06 00:51:02.513931958 +0000 UTC m=+24.843290811"
Sep 06 00:51:02.514141 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.514115    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-5.ec2.internal" podStartSLOduration=23.514109576 podStartE2EDuration="23.514109576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:01.507809353 +0000 UTC m=+23.837168217" watchObservedRunningTime="2026-09-06 00:51:02.514109576 +0000 UTC m=+24.843468414"
Sep 06 00:51:02.760293 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.760248    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:51:02.760950 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:02.760927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:51:03.196265 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:03.196182    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:03.196265 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:03.196219    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:03.196265 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:03.196219    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:03.196525 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:03.196292    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:03.196525 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:03.196395    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:03.196525 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:03.196481    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:04.415711 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:04.415674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:04.518864 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:04.518808    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:04.763283 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:04.755377    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:04.816240 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:04.816212    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:05.195923 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.195752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:05.196024 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.195799    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:05.196024 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:05.196001    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:05.196130 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:05.196100    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:05.196130 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.195813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:05.196213 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:05.196197    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:05.510225 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.510201    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:51:05.510643 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.510555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"a1920e3d749dd2ae576fa3ec9fc24dfcda9fc21cc234eaa92068b5f37ce5b803"}
Sep 06 00:51:05.510945 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.510892    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:51:05.511075 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.510971    2575 scope.go:117] "RemoveContainer" containerID="9836122ce742b472dc27395e9729a488559e561b657b07873707715cb793f879"
Sep 06 00:51:05.511075 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.510990    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:51:05.511075 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.511040    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:51:05.512699 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.512676    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b7afb0c-24a3-4f9f-a295-e1d8c426e03c" containerID="9749cd978452744be32d97c0670333ee8ed53a4c493689a115ab9161460fff6a" exitCode=0
Sep 06 00:51:05.512801 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.512708    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerDied","Data":"9749cd978452744be32d97c0670333ee8ed53a4c493689a115ab9161460fff6a"}
Sep 06 00:51:05.530739 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.530717    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:51:05.530838 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.530821    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:51:05.670982 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.670947    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:05.772444 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.772350    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:05.872204 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.872170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:05.897176 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.897145    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:51:05.897468 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.897277    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 06 00:51:05.897726 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.897711    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2csv8"
Sep 06 00:51:05.975976 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:05.975940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:06.518427 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.518385    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:51:06.518987 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.518921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" event={"ID":"239122a3-2aee-4587-b6ea-694cf69928cd","Type":"ContainerStarted","Data":"6c12993dbc911d767a5449595472df353eadcc2b75b99e080d7839b8dd6b0e83"}
Sep 06 00:51:06.546109 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.545966    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4" podStartSLOduration=9.167418546 podStartE2EDuration="28.545945628s" totalImagesPullingTime="19.378527082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:06.545373625 +0000 UTC m=+28.874732473" watchObservedRunningTime="2026-09-06 00:51:06.545945628 +0000 UTC m=+28.875304467"
Sep 06 00:51:06.599203 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.599176    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xsbhl"]
Sep 06 00:51:06.599350 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.599306    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:06.599452 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:06.599427    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:06.602384 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.602359    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cd6xc"]
Sep 06 00:51:06.602494 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.602454    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:06.602569 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:06.602549    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:06.602993 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.602974    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2nfb"]
Sep 06 00:51:06.603082 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:06.603068    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:06.603179 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:06.603157    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:07.509124 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:07.509090    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:07.522829 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:07.522801    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b7afb0c-24a3-4f9f-a295-e1d8c426e03c" containerID="b45fdeb97d54197fb98e0f9fdf1953bac4e6cdf3a594c99abe0f06f8a72f7e4f" exitCode=0
Sep 06 00:51:07.523242 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:07.522897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerDied","Data":"b45fdeb97d54197fb98e0f9fdf1953bac4e6cdf3a594c99abe0f06f8a72f7e4f"}
Sep 06 00:51:07.568670 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:07.568642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:07.619905 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:07.619713    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.083839 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.083806    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.162483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.162389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.197130 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.197097    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:08.197358 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:08.197181    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:08.197358 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.197278    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:08.197490 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:08.197379    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:08.197490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.197410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:08.197490 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:08.197465    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:08.222080 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.222053    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.306978 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.306943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.372167 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.372141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.775203 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.775177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:08.875572 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:08.875527    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:09.529245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:09.529208    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b7afb0c-24a3-4f9f-a295-e1d8c426e03c" containerID="9571d9b8c18a01fd8914bd7fc66d942b8128596067f2651eaa5dccadd9edc593" exitCode=0
Sep 06 00:51:09.529406 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:09.529281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerDied","Data":"9571d9b8c18a01fd8914bd7fc66d942b8128596067f2651eaa5dccadd9edc593"}
Sep 06 00:51:10.065479 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.065454    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pjw4s_4141a1cc-d62d-4397-9b44-097d6e0f3229/dns-node-resolver/0.log"
Sep 06 00:51:10.195772 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.195740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:10.195772 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.195749    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:10.195997 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.195745    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:10.195997 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.195903    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:10.196085 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.196030    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:10.196149 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.196130    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:10.408382 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.408285    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:10.509508 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.509471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:10.742727 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.742674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:10.940474 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.940435    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:10.940474 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:10.940477    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:10.940727 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.940600    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:51:10.940727 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.940654    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs podName:e887d0d0-952c-470f-bf00-4664435b1043 nodeName:}" failed. No retries permitted until 2026-09-06 00:51:42.940641118 +0000 UTC m=+65.269999932 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs") pod "network-metrics-daemon-g2nfb" (UID: "e887d0d0-952c-470f-bf00-4664435b1043") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 00:51:10.940727 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.940600    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 00:51:10.940727 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:10.940716    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret podName:c5308096-acd0-49da-9d37-ef559fd0e363 nodeName:}" failed. No retries permitted until 2026-09-06 00:51:42.940703629 +0000 UTC m=+65.270062457 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret") pod "global-pull-secret-syncer-xsbhl" (UID: "c5308096-acd0-49da-9d37-ef559fd0e363") : object "kube-system"/"original-pull-secret" not registered
Sep 06 00:51:11.041642 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:11.041552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:11.041792 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:11.041734    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 00:51:11.041792 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:11.041761    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 00:51:11.041792 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:11.041772    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lz5xb for pod openshift-network-diagnostics/network-check-target-cd6xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:51:11.041995 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:11.041848    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb podName:0bf3e9d3-ef39-4c7e-8301-5501b884c860 nodeName:}" failed. No retries permitted until 2026-09-06 00:51:43.041821442 +0000 UTC m=+65.371180276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lz5xb" (UniqueName: "kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb") pod "network-check-target-cd6xc" (UID: "0bf3e9d3-ef39-4c7e-8301-5501b884c860") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 00:51:11.067081 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:11.067056    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b9r8m_16c172de-2c01-4b51-a2ad-3e0b7b35ff3d/node-ca/0.log"
Sep 06 00:51:11.332109 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:11.331990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:12.181846 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.181737    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:12.195890 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.195846    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:12.195890 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.195873    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:12.196080 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:12.195971    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsbhl" podUID="c5308096-acd0-49da-9d37-ef559fd0e363"
Sep 06 00:51:12.196134 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:12.196089    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2nfb" podUID="e887d0d0-952c-470f-bf00-4664435b1043"
Sep 06 00:51:12.196189 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.196155    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:12.196277 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:12.196252    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cd6xc" podUID="0bf3e9d3-ef39-4c7e-8301-5501b884c860"
Sep 06 00:51:12.359092 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.359060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:12.413755 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.413721    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-5.ec2.internal" event="NodeReady"
Sep 06 00:51:12.413936 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.413899    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 06 00:51:12.459120 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.459092    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:12.558900 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.558846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:12.658329 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.658287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:12.883652 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:12.883581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:13.476140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:13.476103    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:13.541781 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:13.541750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerStarted","Data":"7bfd2f39f2695e8fb8af57ec54bcd69425f2c3f05c036f3f52e0601ba6b319dc"}
Sep 06 00:51:13.875523 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:13.875490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:14.195686 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.195605    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:14.195686 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.195633    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:14.195908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.195611    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:14.198655 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.198636    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bctds\""
Sep 06 00:51:14.198655 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.198654    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 00:51:14.198790 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.198736    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7ttx4\""
Sep 06 00:51:14.198790 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.198658    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 00:51:14.198790 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.198778    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 00:51:14.198972 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.198956    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 00:51:14.482529 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.482495    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:14.546130 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.546103    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b7afb0c-24a3-4f9f-a295-e1d8c426e03c" containerID="7bfd2f39f2695e8fb8af57ec54bcd69425f2c3f05c036f3f52e0601ba6b319dc" exitCode=0
Sep 06 00:51:14.546247 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.546142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerDied","Data":"7bfd2f39f2695e8fb8af57ec54bcd69425f2c3f05c036f3f52e0601ba6b319dc"}
Sep 06 00:51:14.806098 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:14.806016    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.073539 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.073305    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.182121 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.182088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.282530 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.282496    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.345493 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.345430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.474971 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.474935    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.551002 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.550969    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b7afb0c-24a3-4f9f-a295-e1d8c426e03c" containerID="66c18ff376524e5db6b1c7a4acbf8ebd29e154e05a063cf267d658affc6d90f1" exitCode=0
Sep 06 00:51:15.551349 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.551016    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerDied","Data":"66c18ff376524e5db6b1c7a4acbf8ebd29e154e05a063cf267d658affc6d90f1"}
Sep 06 00:51:15.871406 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.871329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:15.922779 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:15.922752    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:16.272453 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.272423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:16.474186 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.474150    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:16.556412 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.556384    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7txcq" event={"ID":"6b7afb0c-24a3-4f9f-a295-e1d8c426e03c","Type":"ContainerStarted","Data":"7002d4f7ee727979ac24cc4f03e00d2afd9b7b2d01c8966491438af236b821f3"}
Sep 06 00:51:16.576480 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.576439    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7txcq" podStartSLOduration=8.857081888 podStartE2EDuration="38.576424618s" totalImagesPullingTime="29.71934273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:16.575746103 +0000 UTC m=+38.905104954" watchObservedRunningTime="2026-09-06 00:51:16.576424618 +0000 UTC m=+38.905783454"
Sep 06 00:51:16.739256 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.739188    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:16.843899 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.843848    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:16.940409 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:16.940382    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:17.073310 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:17.073232    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:17.472395 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:17.472359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:18.071436 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:18.071402    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:18.464868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:18.464810    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:18.634545 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:18.634515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:18.874789 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:18.874701    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:19.073996 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:19.073960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:19.687807 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:19.687771    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:19.872734 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:19.872706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:20.279901 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:20.279846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:21.070748 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:21.070721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:21.473291 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:21.473260    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:21.670069 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:21.670038    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:21.870151 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:21.870077    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:21.997605 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:21.997573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:22.098576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:22.098548    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:22.158586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:22.158519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:22.675469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:22.675432    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:22.871687 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:22.871657    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:23.190389 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:23.190353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:30.177631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:30.177596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:30.278908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:30.278872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:33.815873 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:33.815834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:33.972281 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:33.972248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:34.393544 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:34.393512    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:35.264579 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.264548    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"]
Sep 06 00:51:35.266572 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.266556    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.269729 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.269705    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 06 00:51:35.269919 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.269893    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tfhmt\""
Sep 06 00:51:35.271287 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.271270    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 06 00:51:35.271585 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.271565    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 06 00:51:35.278020 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.278002    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"]
Sep 06 00:51:35.279736 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.279722    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.282885 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.282658    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 06 00:51:35.287262 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.287239    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 06 00:51:35.287262 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.287249    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 06 00:51:35.287410 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.287316    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 06 00:51:35.287410 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.287382    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 06 00:51:35.287746 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.287702    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xhndw\""
Sep 06 00:51:35.292107 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.292086    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"]
Sep 06 00:51:35.294789 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.294767    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"]
Sep 06 00:51:35.316497 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.316472    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"]
Sep 06 00:51:35.318150 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.318136    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:35.320759 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.320742    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 06 00:51:35.322148 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.322128    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gflfq"]
Sep 06 00:51:35.323740 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.323723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.326426 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.326409    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 06 00:51:35.326530 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.326512    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-766s4\""
Sep 06 00:51:35.326674 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.326660    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sw7jk\""
Sep 06 00:51:35.326767 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.326750    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 06 00:51:35.334846 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.334827    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"]
Sep 06 00:51:35.339165 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.339124    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gflfq"]
Sep 06 00:51:35.345528 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.345501    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"]
Sep 06 00:51:35.345675 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:51:35.345657    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-fhqzt registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt" podUID="6b70e9e9-80fc-4660-a96f-9de8fc328bc5"
Sep 06 00:51:35.408177 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408145    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-bound-sa-token\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408177 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhqzt\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-kube-api-access-fhqzt\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408362 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-ca-trust-extracted\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408362 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-installation-pull-secrets\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408362 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeb1d6d8-1462-452a-9b63-9acd308fb828-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78b64dbfb-h65rc\" (UID: \"aeb1d6d8-1462-452a-9b63-9acd308fb828\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.408362 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-tls\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-trusted-ca\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b238b9b4-70cc-411d-ad91-b0070261b17e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-drf4v\" (UID: \"b238b9b4-70cc-411d-ad91-b0070261b17e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:35.408586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408419    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qxj4\" (UniqueName: \"kubernetes.io/projected/aeb1d6d8-1462-452a-9b63-9acd308fb828-kube-api-access-9qxj4\") pod \"managed-serviceaccount-addon-agent-78b64dbfb-h65rc\" (UID: \"aeb1d6d8-1462-452a-9b63-9acd308fb828\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.408586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-image-registry-private-configuration\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.408586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.408471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-certificates\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.415379 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.415352    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-65gvx"]
Sep 06 00:51:35.417215 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.417201    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.419568 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.419549    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 06 00:51:35.421487 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.421467    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zgg8h"]
Sep 06 00:51:35.423815 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.423794    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.423911 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.423840    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65d89dfff8-t7m85"]
Sep 06 00:51:35.424649 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.424631    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 06 00:51:35.424739 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.424718    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 06 00:51:35.424812 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.424792    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:35.424930 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.424875    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ht4cf\""
Sep 06 00:51:35.425795 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.425780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.426597 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.426580    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 06 00:51:35.426664 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.426645    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 06 00:51:35.426888 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.426873    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g58m5\""
Sep 06 00:51:35.427362 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.427346    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 06 00:51:35.427681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.427668    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 06 00:51:35.433429 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.433410    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-65gvx"]
Sep 06 00:51:35.441749 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.441730    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgg8h"]
Sep 06 00:51:35.447448 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.447430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65d89dfff8-t7m85"]
Sep 06 00:51:35.509609 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10056f2c-e664-48ea-996d-a50e0dd2d41d-cert\") pod \"ingress-canary-65gvx\" (UID: \"10056f2c-e664-48ea-996d-a50e0dd2d41d\") " pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.509609 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509606    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12e27939-fcf7-48ac-81f0-72a2bb057888-config-volume\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.509835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509636    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-ca-trust-extracted\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.509835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509652    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-installation-pull-secrets\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.509835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeb1d6d8-1462-452a-9b63-9acd308fb828-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78b64dbfb-h65rc\" (UID: \"aeb1d6d8-1462-452a-9b63-9acd308fb828\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.509835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509773    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-tls\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.509835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509795    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-trusted-ca\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.509835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b238b9b4-70cc-411d-ad91-b0070261b17e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-drf4v\" (UID: \"b238b9b4-70cc-411d-ad91-b0070261b17e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:35.510157 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509871    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12e27939-fcf7-48ac-81f0-72a2bb057888-metrics-tls\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.510157 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.509909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e27939-fcf7-48ac-81f0-72a2bb057888-tmp-dir\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.510157 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-ca-trust-extracted\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.510305 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510229    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq2px\" (UniqueName: \"kubernetes.io/projected/12e27939-fcf7-48ac-81f0-72a2bb057888-kube-api-access-vq2px\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.510417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510320    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qxj4\" (UniqueName: \"kubernetes.io/projected/aeb1d6d8-1462-452a-9b63-9acd308fb828-kube-api-access-9qxj4\") pod \"managed-serviceaccount-addon-agent-78b64dbfb-h65rc\" (UID: \"aeb1d6d8-1462-452a-9b63-9acd308fb828\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.510417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64652\" (UniqueName: \"kubernetes.io/projected/10056f2c-e664-48ea-996d-a50e0dd2d41d-kube-api-access-64652\") pod \"ingress-canary-65gvx\" (UID: \"10056f2c-e664-48ea-996d-a50e0dd2d41d\") " pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.510417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-image-registry-private-configuration\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.510417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-certificates\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.510631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510424    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-bound-sa-token\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.510631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhqzt\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-kube-api-access-fhqzt\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.510882 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.510843    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-trusted-ca\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.511310 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.511291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-certificates\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.514679 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.514623    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-tls\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.514679 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.514664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-image-registry-private-configuration\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.514893 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.514873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b238b9b4-70cc-411d-ad91-b0070261b17e-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-drf4v\" (UID: \"b238b9b4-70cc-411d-ad91-b0070261b17e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:35.514960 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.514930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aeb1d6d8-1462-452a-9b63-9acd308fb828-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78b64dbfb-h65rc\" (UID: \"aeb1d6d8-1462-452a-9b63-9acd308fb828\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.515366 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.515347    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-installation-pull-secrets\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.525738 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.525716    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhqzt\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-kube-api-access-fhqzt\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.526085 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.526058    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qxj4\" (UniqueName: \"kubernetes.io/projected/aeb1d6d8-1462-452a-9b63-9acd308fb828-kube-api-access-9qxj4\") pod \"managed-serviceaccount-addon-agent-78b64dbfb-h65rc\" (UID: \"aeb1d6d8-1462-452a-9b63-9acd308fb828\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.526494 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.526470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:35.526600 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.526578    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-bound-sa-token\") pod \"image-registry-59c7dbd9ff-m8jzt\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") " pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.596694 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.596672    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"
Sep 06 00:51:35.598041 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.598021    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.604217 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.604202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:35.611419 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e72cab94-f27f-4ce6-a5a8-95586bdc0955-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.611528 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvf56\" (UniqueName: \"kubernetes.io/projected/e72cab94-f27f-4ce6-a5a8-95586bdc0955-kube-api-access-gvf56\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.611528 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e27939-fcf7-48ac-81f0-72a2bb057888-tmp-dir\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.611645 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq2px\" (UniqueName: \"kubernetes.io/projected/12e27939-fcf7-48ac-81f0-72a2bb057888-kube-api-access-vq2px\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.611645 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwcb8\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-kube-api-access-pwcb8\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611645 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611571    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e72cab94-f27f-4ce6-a5a8-95586bdc0955-crio-socket\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.611645 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-ca-trust-extracted\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611645 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-installation-pull-secrets\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611645 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-registry-certificates\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-bound-sa-token\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611684    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-image-registry-private-configuration\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-registry-tls\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611726    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12e27939-fcf7-48ac-81f0-72a2bb057888-metrics-tls\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611749    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64652\" (UniqueName: \"kubernetes.io/projected/10056f2c-e664-48ea-996d-a50e0dd2d41d-kube-api-access-64652\") pod \"ingress-canary-65gvx\" (UID: \"10056f2c-e664-48ea-996d-a50e0dd2d41d\") " pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12e27939-fcf7-48ac-81f0-72a2bb057888-config-volume\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10056f2c-e664-48ea-996d-a50e0dd2d41d-cert\") pod \"ingress-canary-65gvx\" (UID: \"10056f2c-e664-48ea-996d-a50e0dd2d41d\") " pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611826    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e72cab94-f27f-4ce6-a5a8-95586bdc0955-data-volume\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-trusted-ca\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e27939-fcf7-48ac-81f0-72a2bb057888-tmp-dir\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.611975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.611888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e72cab94-f27f-4ce6-a5a8-95586bdc0955-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.612865 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.612824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12e27939-fcf7-48ac-81f0-72a2bb057888-config-volume\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.614432 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.614410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12e27939-fcf7-48ac-81f0-72a2bb057888-metrics-tls\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.614492 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.614449    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10056f2c-e664-48ea-996d-a50e0dd2d41d-cert\") pod \"ingress-canary-65gvx\" (UID: \"10056f2c-e664-48ea-996d-a50e0dd2d41d\") " pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.622240 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.622214    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:35.624791 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.624772    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64652\" (UniqueName: \"kubernetes.io/projected/10056f2c-e664-48ea-996d-a50e0dd2d41d-kube-api-access-64652\") pod \"ingress-canary-65gvx\" (UID: \"10056f2c-e664-48ea-996d-a50e0dd2d41d\") " pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.624913 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.624896    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq2px\" (UniqueName: \"kubernetes.io/projected/12e27939-fcf7-48ac-81f0-72a2bb057888-kube-api-access-vq2px\") pod \"dns-default-gflfq\" (UID: \"12e27939-fcf7-48ac-81f0-72a2bb057888\") " pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.626750 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.626738    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:35.632293 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.632278    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:35.692906 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.692879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:35.712281 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712258    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhqzt\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-kube-api-access-fhqzt\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712415 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712295    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-installation-pull-secrets\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712415 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712335    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-trusted-ca\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712415 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712370    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-ca-trust-extracted\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712415 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712401    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-tls\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712609 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712440    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-bound-sa-token\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712609 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712466    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-image-registry-private-configuration\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712609 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712514    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-certificates\") pod \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\" (UID: \"6b70e9e9-80fc-4660-a96f-9de8fc328bc5\") "
Sep 06 00:51:35.712758 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712636    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e72cab94-f27f-4ce6-a5a8-95586bdc0955-data-volume\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.712758 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-trusted-ca\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.712758 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712692    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e72cab94-f27f-4ce6-a5a8-95586bdc0955-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.712758 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e72cab94-f27f-4ce6-a5a8-95586bdc0955-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.712975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvf56\" (UniqueName: \"kubernetes.io/projected/e72cab94-f27f-4ce6-a5a8-95586bdc0955-kube-api-access-gvf56\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.712975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712821    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:51:35.712975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwcb8\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-kube-api-access-pwcb8\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.712975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e72cab94-f27f-4ce6-a5a8-95586bdc0955-crio-socket\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.712975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712917    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-ca-trust-extracted\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.712975 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712954    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-installation-pull-secrets\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.713245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.712981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-registry-certificates\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.713245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.713012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-bound-sa-token\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.713245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.713048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-image-registry-private-configuration\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.713245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.713074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-registry-tls\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.713245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.713228    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:51:35.713837 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.713690    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e72cab94-f27f-4ce6-a5a8-95586bdc0955-crio-socket\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.713977 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.713944    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-ca-trust-extracted\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.714664 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.714274    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:51:35.715116 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.715094    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e72cab94-f27f-4ce6-a5a8-95586bdc0955-data-volume\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.715491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.715467    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-registry-certificates\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.715568 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.715468    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e72cab94-f27f-4ce6-a5a8-95586bdc0955-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.715795 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.715778    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-trusted-ca\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.716258 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.716216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-registry-tls\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.717556 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.717523    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-image-registry-private-configuration\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.718184 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718159    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:51:35.718184 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-kube-api-access-fhqzt" (OuterVolumeSpecName: "kube-api-access-fhqzt") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "kube-api-access-fhqzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:51:35.718303 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718239    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:51:35.718303 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718249    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:51:35.718371 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6b70e9e9-80fc-4660-a96f-9de8fc328bc5" (UID: "6b70e9e9-80fc-4660-a96f-9de8fc328bc5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:51:35.718371 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718329    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e72cab94-f27f-4ce6-a5a8-95586bdc0955-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.719017 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.718997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-installation-pull-secrets\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.722226 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.722206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-bound-sa-token\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.722688 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.722659    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwcb8\" (UniqueName: \"kubernetes.io/projected/1b5fdad7-cc5f-424e-af51-6a5b2fa77701-kube-api-access-pwcb8\") pod \"image-registry-65d89dfff8-t7m85\" (UID: \"1b5fdad7-cc5f-424e-af51-6a5b2fa77701\") " pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.723233 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.723213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvf56\" (UniqueName: \"kubernetes.io/projected/e72cab94-f27f-4ce6-a5a8-95586bdc0955-kube-api-access-gvf56\") pod \"insights-runtime-extractor-zgg8h\" (UID: \"e72cab94-f27f-4ce6-a5a8-95586bdc0955\") " pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.726126 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.726114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-65gvx"
Sep 06 00:51:35.734627 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.734608    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgg8h"
Sep 06 00:51:35.739328 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.739312    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:35.789908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.789841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:35.813624 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813597    2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-certificates\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813629    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhqzt\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-kube-api-access-fhqzt\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813646    2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-installation-pull-secrets\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813661    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-trusted-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813675    2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-ca-trust-extracted\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813687    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-registry-tls\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813701    2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-bound-sa-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.813717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.813714    2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b70e9e9-80fc-4660-a96f-9de8fc328bc5-image-registry-private-configuration\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:51:35.881409 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.881382    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc"]
Sep 06 00:51:35.884716 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:35.884619    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaeb1d6d8_1462_452a_9b63_9acd308fb828.slice/crio-361a16d2e83d6d4b0607823e252f2b09f750360ce8713aa3bfd196449a7b5a1c WatchSource:0}: Error finding container 361a16d2e83d6d4b0607823e252f2b09f750360ce8713aa3bfd196449a7b5a1c: Status 404 returned error can't find the container with id 361a16d2e83d6d4b0607823e252f2b09f750360ce8713aa3bfd196449a7b5a1c
Sep 06 00:51:35.903482 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.903457    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"]
Sep 06 00:51:35.906473 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:35.906448    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb238b9b4_70cc_411d_ad91_b0070261b17e.slice/crio-4356d6e2c83b71c0702b1863f1e0a8a88359c85dc4fbf017143a02d5317f7c7e WatchSource:0}: Error finding container 4356d6e2c83b71c0702b1863f1e0a8a88359c85dc4fbf017143a02d5317f7c7e: Status 404 returned error can't find the container with id 4356d6e2c83b71c0702b1863f1e0a8a88359c85dc4fbf017143a02d5317f7c7e
Sep 06 00:51:35.921398 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.921351    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gflfq"]
Sep 06 00:51:35.924001 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:35.923982    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e27939_fcf7_48ac_81f0_72a2bb057888.slice/crio-d717fc241b037104e3a93ca551beb769a0e031dfedd2608e7737cf99ccf678a5 WatchSource:0}: Error finding container d717fc241b037104e3a93ca551beb769a0e031dfedd2608e7737cf99ccf678a5: Status 404 returned error can't find the container with id d717fc241b037104e3a93ca551beb769a0e031dfedd2608e7737cf99ccf678a5
Sep 06 00:51:35.999033 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:35.999004    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-65gvx"]
Sep 06 00:51:36.002377 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:36.002338    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10056f2c_e664_48ea_996d_a50e0dd2d41d.slice/crio-52bb0faacb33ea6ff5a2a70c2dea639bb05b94d9a80cf27fc0b0d9305ccbe036 WatchSource:0}: Error finding container 52bb0faacb33ea6ff5a2a70c2dea639bb05b94d9a80cf27fc0b0d9305ccbe036: Status 404 returned error can't find the container with id 52bb0faacb33ea6ff5a2a70c2dea639bb05b94d9a80cf27fc0b0d9305ccbe036
Sep 06 00:51:36.010788 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.010759    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgg8h"]
Sep 06 00:51:36.014147 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:36.014121    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72cab94_f27f_4ce6_a5a8_95586bdc0955.slice/crio-09f320f4920427b9217d2a3bd40d7e35a75df731ec16f171e5bbedf378f130da WatchSource:0}: Error finding container 09f320f4920427b9217d2a3bd40d7e35a75df731ec16f171e5bbedf378f130da: Status 404 returned error can't find the container with id 09f320f4920427b9217d2a3bd40d7e35a75df731ec16f171e5bbedf378f130da
Sep 06 00:51:36.026661 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.026602    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65d89dfff8-t7m85"]
Sep 06 00:51:36.030671 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:36.030643    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5fdad7_cc5f_424e_af51_6a5b2fa77701.slice/crio-d455821031eb85ba7e03f1c9406bcc03864fe3a956e4055f9e6adb0aff87fe04 WatchSource:0}: Error finding container d455821031eb85ba7e03f1c9406bcc03864fe3a956e4055f9e6adb0aff87fe04: Status 404 returned error can't find the container with id d455821031eb85ba7e03f1c9406bcc03864fe3a956e4055f9e6adb0aff87fe04
Sep 06 00:51:36.165024 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.164994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:36.227330 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.227297    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:36.602627 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.602594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gflfq" event={"ID":"12e27939-fcf7-48ac-81f0-72a2bb057888","Type":"ContainerStarted","Data":"d717fc241b037104e3a93ca551beb769a0e031dfedd2608e7737cf99ccf678a5"}
Sep 06 00:51:36.604615 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.604585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc" event={"ID":"aeb1d6d8-1462-452a-9b63-9acd308fb828","Type":"ContainerStarted","Data":"361a16d2e83d6d4b0607823e252f2b09f750360ce8713aa3bfd196449a7b5a1c"}
Sep 06 00:51:36.606567 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.606540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85" event={"ID":"1b5fdad7-cc5f-424e-af51-6a5b2fa77701","Type":"ContainerStarted","Data":"9d45f5a30f619719286fef5a6ae349e94fa2ca832d06cc0e90fd49c49f4ad2ae"}
Sep 06 00:51:36.606677 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.606590    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85" event={"ID":"1b5fdad7-cc5f-424e-af51-6a5b2fa77701","Type":"ContainerStarted","Data":"d455821031eb85ba7e03f1c9406bcc03864fe3a956e4055f9e6adb0aff87fe04"}
Sep 06 00:51:36.606962 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.606942    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:36.608901 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.608879    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v" event={"ID":"b238b9b4-70cc-411d-ad91-b0070261b17e","Type":"ContainerStarted","Data":"4356d6e2c83b71c0702b1863f1e0a8a88359c85dc4fbf017143a02d5317f7c7e"}
Sep 06 00:51:36.612141 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.612057    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgg8h" event={"ID":"e72cab94-f27f-4ce6-a5a8-95586bdc0955","Type":"ContainerStarted","Data":"ac6569e81ab5f1660f31744bc9198989709b69204ad0564019ba9d2812b63384"}
Sep 06 00:51:36.612141 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.612107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgg8h" event={"ID":"e72cab94-f27f-4ce6-a5a8-95586bdc0955","Type":"ContainerStarted","Data":"09f320f4920427b9217d2a3bd40d7e35a75df731ec16f171e5bbedf378f130da"}
Sep 06 00:51:36.613631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.613358    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-65gvx" event={"ID":"10056f2c-e664-48ea-996d-a50e0dd2d41d","Type":"ContainerStarted","Data":"52bb0faacb33ea6ff5a2a70c2dea639bb05b94d9a80cf27fc0b0d9305ccbe036"}
Sep 06 00:51:36.613631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.613376    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"
Sep 06 00:51:36.627915 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.626343    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85" podStartSLOduration=1.626325718 podStartE2EDuration="1.626325718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:36.625093188 +0000 UTC m=+58.954452024" watchObservedRunningTime="2026-09-06 00:51:36.626325718 +0000 UTC m=+58.955684556"
Sep 06 00:51:36.655907 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.655881    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"]
Sep 06 00:51:36.671838 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:36.671804    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59c7dbd9ff-m8jzt"]
Sep 06 00:51:37.550430 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:37.550395    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmkb4"
Sep 06 00:51:37.708437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:37.708366    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:37.766321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:37.766286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:38.199176 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:38.199149    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b70e9e9-80fc-4660-a96f-9de8fc328bc5" path="/var/lib/kubelet/pods/6b70e9e9-80fc-4660-a96f-9de8fc328bc5/volumes"
Sep 06 00:51:40.630916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.630823    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v" event={"ID":"b238b9b4-70cc-411d-ad91-b0070261b17e","Type":"ContainerStarted","Data":"ef4b8c1269a9a9c9dcb8c7bc2f173352d903f5facdc6d70f6c3ea36b5aeb919d"}
Sep 06 00:51:40.631377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.631004    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:40.633672 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.633648    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgg8h" event={"ID":"e72cab94-f27f-4ce6-a5a8-95586bdc0955","Type":"ContainerStarted","Data":"dd721d8ab5d1b5c71897b50cfe93f49a97b17f676261c252d64b7761009cb30d"}
Sep 06 00:51:40.635371 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.635347    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-65gvx" event={"ID":"10056f2c-e664-48ea-996d-a50e0dd2d41d","Type":"ContainerStarted","Data":"60723f2d736ba48b64077ae64c775d15a8b6838972d1606ab92849dd704953ac"}
Sep 06 00:51:40.636658 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.636625    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v"
Sep 06 00:51:40.638046 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.637568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gflfq" event={"ID":"12e27939-fcf7-48ac-81f0-72a2bb057888","Type":"ContainerStarted","Data":"23c2265465fb7a533e7f104b793733b65a3e8f97f0757d48aec11c7060910725"}
Sep 06 00:51:40.638046 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.637610    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gflfq" event={"ID":"12e27939-fcf7-48ac-81f0-72a2bb057888","Type":"ContainerStarted","Data":"caed524eee872e3c144615dbe92a6b793f5a73510a167376355a4329c2cce238"}
Sep 06 00:51:40.638046 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.637637    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:40.639380 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.639363    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc" event={"ID":"aeb1d6d8-1462-452a-9b63-9acd308fb828","Type":"ContainerStarted","Data":"4521411293d684ba581973feeb23853cc640321d2cf843b7b1e7e3d8ab1e917b"}
Sep 06 00:51:40.664483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.664438    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-drf4v" podStartSLOduration=1.558541263 podStartE2EDuration="5.664422439s" totalImagesPullingTime="4.105881176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:40.649980796 +0000 UTC m=+62.979339643" watchObservedRunningTime="2026-09-06 00:51:40.664422439 +0000 UTC m=+62.993781276"
Sep 06 00:51:40.683485 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.683445    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gflfq" podStartSLOduration=1.590065885 podStartE2EDuration="5.683432582s" totalImagesPullingTime="4.093366697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:40.683429869 +0000 UTC m=+63.012788726" watchObservedRunningTime="2026-09-06 00:51:40.683432582 +0000 UTC m=+63.012791412"
Sep 06 00:51:40.700669 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.700623    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-65gvx" podStartSLOduration=1.6836551050000002 podStartE2EDuration="5.700605921s" totalImagesPullingTime="4.016950816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:40.699684391 +0000 UTC m=+63.029043249" watchObservedRunningTime="2026-09-06 00:51:40.700605921 +0000 UTC m=+63.029964748"
Sep 06 00:51:40.720360 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:40.720131    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78b64dbfb-h65rc" podStartSLOduration=1.582071758 podStartE2EDuration="5.720118917s" totalImagesPullingTime="4.138047159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:40.719002307 +0000 UTC m=+63.048361142" watchObservedRunningTime="2026-09-06 00:51:40.720118917 +0000 UTC m=+63.049477847"
Sep 06 00:51:41.004728 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:41.004692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:41.072504 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:41.072469    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:41.163330 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:41.163297    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:41.647067 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:41.646961    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgg8h" event={"ID":"e72cab94-f27f-4ce6-a5a8-95586bdc0955","Type":"ContainerStarted","Data":"b81049a5461549f9dd917cddf7b9625a2da8f383e36c5bffd3c6e72caa543733"}
Sep 06 00:51:41.666694 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:41.666649    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zgg8h" podStartSLOduration=1.623032722 podStartE2EDuration="6.666635438s" totalImagesPullingTime="5.043602716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:51:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:41.665780022 +0000 UTC m=+63.995138875" watchObservedRunningTime="2026-09-06 00:51:41.666635438 +0000 UTC m=+63.995994274"
Sep 06 00:51:42.713799 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.713770    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:42.814500 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.814464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:42.866148 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.866121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:42.945247 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.945208    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:42.976338 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.976263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:42.976338 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.976328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:42.978867 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.978840    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 00:51:42.978956 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.978908    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 00:51:42.989001 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.988981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5308096-acd0-49da-9d37-ef559fd0e363-original-pull-secret\") pod \"global-pull-secret-syncer-xsbhl\" (UID: \"c5308096-acd0-49da-9d37-ef559fd0e363\") " pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:42.989001 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:42.988996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e887d0d0-952c-470f-bf00-4664435b1043-metrics-certs\") pod \"network-metrics-daemon-g2nfb\" (UID: \"e887d0d0-952c-470f-bf00-4664435b1043\") " pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:43.010163 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.010141    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsbhl"
Sep 06 00:51:43.016305 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.016290    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bctds\""
Sep 06 00:51:43.024993 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.024973    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2nfb"
Sep 06 00:51:43.076926 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.076828    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:43.081462 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.081097    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 00:51:43.089613 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.089591    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 00:51:43.101225 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.101144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz5xb\" (UniqueName: \"kubernetes.io/projected/0bf3e9d3-ef39-4c7e-8301-5501b884c860-kube-api-access-lz5xb\") pod \"network-check-target-cd6xc\" (UID: \"0bf3e9d3-ef39-4c7e-8301-5501b884c860\") " pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:43.278913 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.278829    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xsbhl"]
Sep 06 00:51:43.281533 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:43.281509    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5308096_acd0_49da_9d37_ef559fd0e363.slice/crio-41b42fab7a1a6d4bd184658ecf7ca702e15da91f4e0597e2a717dd056355cd40 WatchSource:0}: Error finding container 41b42fab7a1a6d4bd184658ecf7ca702e15da91f4e0597e2a717dd056355cd40: Status 404 returned error can't find the container with id 41b42fab7a1a6d4bd184658ecf7ca702e15da91f4e0597e2a717dd056355cd40
Sep 06 00:51:43.300222 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.300199    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2nfb"]
Sep 06 00:51:43.302770 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:43.302748    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode887d0d0_952c_470f_bf00_4664435b1043.slice/crio-22631eb30f4d40114893d4d5df85ad48cd0155d4f869f8e9e3024694cb5f0f63 WatchSource:0}: Error finding container 22631eb30f4d40114893d4d5df85ad48cd0155d4f869f8e9e3024694cb5f0f63: Status 404 returned error can't find the container with id 22631eb30f4d40114893d4d5df85ad48cd0155d4f869f8e9e3024694cb5f0f63
Sep 06 00:51:43.306690 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.306672    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7ttx4\""
Sep 06 00:51:43.315087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.315072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:43.581134 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.581066    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cd6xc"]
Sep 06 00:51:43.583702 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:43.583666    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf3e9d3_ef39_4c7e_8301_5501b884c860.slice/crio-66b7a1fb67fd1c34003f53ac8b95c914461760d0b7f400f888aa070d4d7b97a1 WatchSource:0}: Error finding container 66b7a1fb67fd1c34003f53ac8b95c914461760d0b7f400f888aa070d4d7b97a1: Status 404 returned error can't find the container with id 66b7a1fb67fd1c34003f53ac8b95c914461760d0b7f400f888aa070d4d7b97a1
Sep 06 00:51:43.654318 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.654290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cd6xc" event={"ID":"0bf3e9d3-ef39-4c7e-8301-5501b884c860","Type":"ContainerStarted","Data":"66b7a1fb67fd1c34003f53ac8b95c914461760d0b7f400f888aa070d4d7b97a1"}
Sep 06 00:51:43.655452 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.655421    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2nfb" event={"ID":"e887d0d0-952c-470f-bf00-4664435b1043","Type":"ContainerStarted","Data":"22631eb30f4d40114893d4d5df85ad48cd0155d4f869f8e9e3024694cb5f0f63"}
Sep 06 00:51:43.656497 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:43.656478    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xsbhl" event={"ID":"c5308096-acd0-49da-9d37-ef559fd0e363","Type":"ContainerStarted","Data":"41b42fab7a1a6d4bd184658ecf7ca702e15da91f4e0597e2a717dd056355cd40"}
Sep 06 00:51:45.155538 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.155505    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.376869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.376823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.405730 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.405701    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-26r2b"]
Sep 06 00:51:45.428323 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.428291    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.432363 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.432341    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 06 00:51:45.432685 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.432668    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 06 00:51:45.432962 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.432940    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 06 00:51:45.433107 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.432945    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 06 00:51:45.433215 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.433196    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pgq27\""
Sep 06 00:51:45.433303 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.433267    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 06 00:51:45.433612 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.433590    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 06 00:51:45.469038 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.469011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.495268 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-sys\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-root\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-metrics-client-ca\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-tls\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld4zp\" (UniqueName: \"kubernetes.io/projected/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-kube-api-access-ld4zp\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-textfile\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-wtmp\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-accelerators-collector-config\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.495656 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.495461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.528195 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.528154    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.596004 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.595977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-sys\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-root\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-metrics-client-ca\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-tls\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596098    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-root\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-sys\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596100    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld4zp\" (UniqueName: \"kubernetes.io/projected/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-kube-api-access-ld4zp\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-textfile\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-wtmp\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-accelerators-collector-config\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596473    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-wtmp\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596878 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-textfile\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.596878 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.596783    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-metrics-client-ca\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.597098 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.597073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-accelerators-collector-config\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.598802 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.598757    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-tls\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.598964 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.598939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.606838 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.606788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld4zp\" (UniqueName: \"kubernetes.io/projected/6c5965a0-d86f-4eaa-9d5f-0b1444b13047-kube-api-access-ld4zp\") pod \"node-exporter-26r2b\" (UID: \"6c5965a0-d86f-4eaa-9d5f-0b1444b13047\") " pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.647667 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.647550    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.667342 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.667310    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2nfb" event={"ID":"e887d0d0-952c-470f-bf00-4664435b1043","Type":"ContainerStarted","Data":"7fee0756d89c171a8f45061c53070dea8793e87ec2f68edc45a5540d218d334c"}
Sep 06 00:51:45.667465 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.667369    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2nfb" event={"ID":"e887d0d0-952c-470f-bf00-4664435b1043","Type":"ContainerStarted","Data":"25cfb71d23bc63fc32fac5cf6701a78c6e822be698bd15767e848057609e305f"}
Sep 06 00:51:45.684571 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.684128    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g2nfb" podStartSLOduration=66.324598323 podStartE2EDuration="1m7.684109289s" totalImagesPullingTime="1.359510966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:45.682754971 +0000 UTC m=+68.012113843" watchObservedRunningTime="2026-09-06 00:51:45.684109289 +0000 UTC m=+68.013468125"
Sep 06 00:51:45.738491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.738463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-26r2b"
Sep 06 00:51:45.748469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.748440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.749790 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:45.749752    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5965a0_d86f_4eaa_9d5f_0b1444b13047.slice/crio-84296357c4c4694a6fa73b9d33940ad74baa2f34cd02e1a4b136b9c3ce943cc6 WatchSource:0}: Error finding container 84296357c4c4694a6fa73b9d33940ad74baa2f34cd02e1a4b136b9c3ce943cc6: Status 404 returned error can't find the container with id 84296357c4c4694a6fa73b9d33940ad74baa2f34cd02e1a4b136b9c3ce943cc6
Sep 06 00:51:45.849925 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.849893    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:45.951153 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:45.951018    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:46.013332 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:46.013296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:46.379091 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:46.379004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:46.461725 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:46.461686    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:46.579987 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:46.579931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:46.672532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:46.672443    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26r2b" event={"ID":"6c5965a0-d86f-4eaa-9d5f-0b1444b13047","Type":"ContainerStarted","Data":"84296357c4c4694a6fa73b9d33940ad74baa2f34cd02e1a4b136b9c3ce943cc6"}
Sep 06 00:51:46.680262 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:46.680234    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:47.539715 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:47.539681    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:47.597397 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:47.597364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:48.295913 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.295880    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:48.371987 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.371952    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:48.375801 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.375775    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"]
Sep 06 00:51:48.378444 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.378426    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.381305 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381281    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 06 00:51:48.381305 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381299    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-54he0um1geud6\""
Sep 06 00:51:48.381456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381286    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 06 00:51:48.381456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381287    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 06 00:51:48.381456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381286    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 06 00:51:48.381456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381446    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xzwdb\""
Sep 06 00:51:48.381616 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.381541    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 06 00:51:48.394330 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.394307    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"]
Sep 06 00:51:48.419465 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419430    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-tls\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419641 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419641 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-metrics-client-ca\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419641 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419523    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-grpc-tls\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419641 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419641 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qkn5\" (UniqueName: \"kubernetes.io/projected/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-kube-api-access-4qkn5\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419919 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.419919 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.419687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.520810 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-tls\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.520893 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.520893 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520838    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-metrics-client-ca\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.520893 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-grpc-tls\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.521039 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520897    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.521039 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qkn5\" (UniqueName: \"kubernetes.io/projected/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-kube-api-access-4qkn5\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.521039 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.520991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.521039 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.521014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.521761 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.521543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-metrics-client-ca\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.524090 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.524038    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.524440 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.524412    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.524543 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.524494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-tls\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.524951 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.524925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-grpc-tls\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.525863 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.525811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.525945 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.525900    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.529520 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.529494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qkn5\" (UniqueName: \"kubernetes.io/projected/e2b097bd-a873-43f2-bbd5-82759cf0b6e0-kube-api-access-4qkn5\") pod \"thanos-querier-6cb79589c5-5v6fk\" (UID: \"e2b097bd-a873-43f2-bbd5-82759cf0b6e0\") " pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.608079 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.607973    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:48.682657 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.682630    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xsbhl" event={"ID":"c5308096-acd0-49da-9d37-ef559fd0e363","Type":"ContainerStarted","Data":"549abd2174d4b1fef7b67d98d2debec1529e48b24522b3876c42f115bfae1d5f"}
Sep 06 00:51:48.684321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.684300    2575 generic.go:356] "Generic (PLEG): container finished" podID="6c5965a0-d86f-4eaa-9d5f-0b1444b13047" containerID="cfd4c89803321c29aa086c609fa7cf5ea5807cf5ad7686891065e97233bf94b3" exitCode=0
Sep 06 00:51:48.684417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.684383    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26r2b" event={"ID":"6c5965a0-d86f-4eaa-9d5f-0b1444b13047","Type":"ContainerDied","Data":"cfd4c89803321c29aa086c609fa7cf5ea5807cf5ad7686891065e97233bf94b3"}
Sep 06 00:51:48.686081 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.686054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cd6xc" event={"ID":"0bf3e9d3-ef39-4c7e-8301-5501b884c860","Type":"ContainerStarted","Data":"fb4ee47a69a0ca0a5fbeaf62af85a3fb5d3a0ab2139b335277ecdbba314ddae5"}
Sep 06 00:51:48.686211 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.686196    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:51:48.688989 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.688975    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:48.698914 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.697478    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xsbhl" podStartSLOduration=65.496446056 podStartE2EDuration="1m10.697461545s" totalImagesPullingTime="5.201015489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:48.697347 +0000 UTC m=+71.026705837" watchObservedRunningTime="2026-09-06 00:51:48.697461545 +0000 UTC m=+71.026820382"
Sep 06 00:51:48.712741 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.712694    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cd6xc" podStartSLOduration=65.81404057 podStartE2EDuration="1m10.712675934s" totalImagesPullingTime="4.898635364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:48.711549207 +0000 UTC m=+71.040908074" watchObservedRunningTime="2026-09-06 00:51:48.712675934 +0000 UTC m=+71.042034771"
Sep 06 00:51:48.971500 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:48.971468    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"]
Sep 06 00:51:48.975503 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:48.975475    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b097bd_a873_43f2_bbd5_82759cf0b6e0.slice/crio-311ad5a1798f7303aac6f6295b59e91dc77213ec4a238346c3452aa850bdcb74 WatchSource:0}: Error finding container 311ad5a1798f7303aac6f6295b59e91dc77213ec4a238346c3452aa850bdcb74: Status 404 returned error can't find the container with id 311ad5a1798f7303aac6f6295b59e91dc77213ec4a238346c3452aa850bdcb74
Sep 06 00:51:49.692093 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:49.692053    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"311ad5a1798f7303aac6f6295b59e91dc77213ec4a238346c3452aa850bdcb74"}
Sep 06 00:51:49.694675 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:49.694639    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26r2b" event={"ID":"6c5965a0-d86f-4eaa-9d5f-0b1444b13047","Type":"ContainerStarted","Data":"1efc17e63030be67bf737ce766dda404f0bbcc3687413605a0ccae9fdd0d5ab9"}
Sep 06 00:51:49.694799 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:49.694690    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26r2b" event={"ID":"6c5965a0-d86f-4eaa-9d5f-0b1444b13047","Type":"ContainerStarted","Data":"1a7246e2cbd043c6ebfd88273dcfa86c8870053926f72ff39779745a23003cec"}
Sep 06 00:51:49.716870 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:49.716808    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-26r2b" podStartSLOduration=1.8801897680000002 podStartE2EDuration="4.716766308s" totalImagesPullingTime="2.761006678s" totalInitContainerRuntime="75.569862ms" isStatefulPod=true podCreationTimestamp="2026-09-06 00:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:49.715739565 +0000 UTC m=+72.045098467" watchObservedRunningTime="2026-09-06 00:51:49.716766308 +0000 UTC m=+72.046125147"
Sep 06 00:51:49.886259 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:49.886224    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:49.987560 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:49.987462    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:50.089079 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:50.089017    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:50.169064 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:50.169029    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:50.607169 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:50.607135    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:50.649835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:50.649807    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gflfq"
Sep 06 00:51:51.634821 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:51.634798    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:51.704282 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:51.704256    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"7ad93ef96622117d5c256926a7fc1897836fecb1d03f8b43c5043ef5ed615b95"}
Sep 06 00:51:51.704360 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:51.704318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"6b3282a4432d6a54e92867110fd21eefd89113c97d55ac845bbfbec379380404"}
Sep 06 00:51:51.704360 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:51.704344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"22a2918088a8b33cec4d42e9cff8e22ee3abb6272422b6171a08dea52fd5881f"}
Sep 06 00:51:51.704432 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:51.704367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"f9281b10b500f0e460c7488fb0d5c6c7880e4ffc97adcf3baec497e1bd900498"}
Sep 06 00:51:52.710478 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:52.710446    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"f44cc0bd5f874972318a368cc7d4b928a4a3252397ec5fcf517e576dbed79019"}
Sep 06 00:51:52.710840 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:52.710521    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" event={"ID":"e2b097bd-a873-43f2-bbd5-82759cf0b6e0","Type":"ContainerStarted","Data":"ef1ed7430466f1279b9c32308644ce26ed57112826806453e40849fc20937782"}
Sep 06 00:51:52.710840 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:52.710570    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:52.733053 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:52.733001    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk" podStartSLOduration=2.2692197800000002 podStartE2EDuration="4.732983467s" totalImagesPullingTime="2.463763687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:51:52.731752947 +0000 UTC m=+75.061111787" watchObservedRunningTime="2026-09-06 00:51:52.732983467 +0000 UTC m=+75.062342302"
Sep 06 00:51:53.027531 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.027441    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:53.127678 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.127642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:53.185928 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.185893    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:53.977481 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.977450    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-tnj9f"]
Sep 06 00:51:53.980728 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.980713    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:51:53.983214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.983193    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jnql7\""
Sep 06 00:51:53.983475 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.983456    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 06 00:51:53.983596 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.983491    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 06 00:51:53.991875 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.991834    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-tnj9f"]
Sep 06 00:51:53.995994 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:53.995963    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.061553 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.061521    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf9gg\" (UniqueName: \"kubernetes.io/projected/cea8e73c-c3ef-416e-ae8d-ef2110193236-kube-api-access-jf9gg\") pod \"downloads-85d98c998f-tnj9f\" (UID: \"cea8e73c-c3ef-416e-ae8d-ef2110193236\") " pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:51:54.071717 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.071694    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.137505 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.137481    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.162669 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.162647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf9gg\" (UniqueName: \"kubernetes.io/projected/cea8e73c-c3ef-416e-ae8d-ef2110193236-kube-api-access-jf9gg\") pod \"downloads-85d98c998f-tnj9f\" (UID: \"cea8e73c-c3ef-416e-ae8d-ef2110193236\") " pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:51:54.170607 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.170584    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf9gg\" (UniqueName: \"kubernetes.io/projected/cea8e73c-c3ef-416e-ae8d-ef2110193236-kube-api-access-jf9gg\") pod \"downloads-85d98c998f-tnj9f\" (UID: \"cea8e73c-c3ef-416e-ae8d-ef2110193236\") " pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:51:54.290257 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.290195    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:51:54.304719 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.304690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.464871 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.464821    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.563273 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.563249    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-tnj9f"]
Sep 06 00:51:54.565549 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:51:54.565515    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea8e73c_c3ef_416e_ae8d_ef2110193236.slice/crio-3a39624fa078648c52546d4a0ced9470ea557d43a1dfe117cb6ec60a029e907e WatchSource:0}: Error finding container 3a39624fa078648c52546d4a0ced9470ea557d43a1dfe117cb6ec60a029e907e: Status 404 returned error can't find the container with id 3a39624fa078648c52546d4a0ced9470ea557d43a1dfe117cb6ec60a029e907e
Sep 06 00:51:54.567730 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.567702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.718956 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.718920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-tnj9f" event={"ID":"cea8e73c-c3ef-416e-ae8d-ef2110193236","Type":"ContainerStarted","Data":"3a39624fa078648c52546d4a0ced9470ea557d43a1dfe117cb6ec60a029e907e"}
Sep 06 00:51:54.887438 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.887357    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:54.988625 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:54.988586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:55.287349 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:55.287311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:55.352179 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:55.352132    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:55.665646 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:55.665553    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:55.964234 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:55.964200    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:56.065242 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:56.065199    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:57.290490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:57.290438    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:57.345627 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:57.345592    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:57.621376 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:57.621310    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65d89dfff8-t7m85"
Sep 06 00:51:57.750416 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:57.750381    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:57.850423 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:57.850387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:58.720427 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:58.720303    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6cb79589c5-5v6fk"
Sep 06 00:51:59.785579 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:59.785542    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:59.885393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:59.885364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:51:59.986042 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:51:59.986009    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:00.046074 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:00.045993    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:00.172325 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:00.172283    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:00.227026 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:00.227002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:02.119756 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:02.119722    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:02.175109 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:02.175071    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:05.518321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.518291    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77fcd9b64d-mblmt"]
Sep 06 00:52:05.524021 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.523935    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.527784 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.527670    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 06 00:52:05.527914 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.527796    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 06 00:52:05.527914 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.527805    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 06 00:52:05.527914 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.527806    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 06 00:52:05.528045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.527938    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pfpwc\""
Sep 06 00:52:05.528045 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.527984    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 06 00:52:05.532062 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.532008    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77fcd9b64d-mblmt"]
Sep 06 00:52:05.652253 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.652215    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-service-ca\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.652253 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.652252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fms8\" (UniqueName: \"kubernetes.io/projected/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-kube-api-access-6fms8\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.652469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.652288    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-serving-cert\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.652469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.652358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-oauth-serving-cert\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.652469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.652393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-config\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.652469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.652436    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-oauth-config\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.752892 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.752845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-config\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.752916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-oauth-config\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.752956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-service-ca\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.752978    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fms8\" (UniqueName: \"kubernetes.io/projected/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-kube-api-access-6fms8\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.753009    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-serving-cert\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.753045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-oauth-serving-cert\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753658 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.753591    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-config\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753771 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.753666    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-service-ca\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.753771 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.753666    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-oauth-serving-cert\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.756092 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.756071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-oauth-config\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.756211 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.756128    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-serving-cert\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.762456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.762436    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fms8\" (UniqueName: \"kubernetes.io/projected/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-kube-api-access-6fms8\") pod \"console-77fcd9b64d-mblmt\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") " pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.779180 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.779112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:05.836136 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.836111    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:05.878644 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:05.878616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:09.047149 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:09.047111    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:09.150321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:09.150285    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:10.698078 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:10.698042    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:10.759577 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:10.759538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:11.500216 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.500130    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77fcd9b64d-mblmt"]
Sep 06 00:52:11.535456 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:52:11.535429    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a960c83_9f68_4545_ae2d_9bb7b3b473e1.slice/crio-c059edd35c3adbc87a0e4369774a3e3fa1bdffde82a8779168a089541d47fd19 WatchSource:0}: Error finding container c059edd35c3adbc87a0e4369774a3e3fa1bdffde82a8779168a089541d47fd19: Status 404 returned error can't find the container with id c059edd35c3adbc87a0e4369774a3e3fa1bdffde82a8779168a089541d47fd19
Sep 06 00:52:11.792138 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.792050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-tnj9f" event={"ID":"cea8e73c-c3ef-416e-ae8d-ef2110193236","Type":"ContainerStarted","Data":"609c95022d4ab672605c557f34c13580a1629043b94a028ce0ff2dbd30bcf4a0"}
Sep 06 00:52:11.792556 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.792206    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:52:11.793384 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.793360    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fcd9b64d-mblmt" event={"ID":"9a960c83-9f68-4545-ae2d-9bb7b3b473e1","Type":"ContainerStarted","Data":"c059edd35c3adbc87a0e4369774a3e3fa1bdffde82a8779168a089541d47fd19"}
Sep 06 00:52:11.793800 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.793779    2575 patch_prober.go:28] interesting pod/downloads-85d98c998f-tnj9f container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.14:8080/\": dial tcp 10.133.0.14:8080: connect: connection refused" start-of-body=
Sep 06 00:52:11.793901 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.793823    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-tnj9f" podUID="cea8e73c-c3ef-416e-ae8d-ef2110193236" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.14:8080/\": dial tcp 10.133.0.14:8080: connect: connection refused"
Sep 06 00:52:11.810452 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:11.809256    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-tnj9f" podStartSLOduration=1.7825008119999999 podStartE2EDuration="18.809234114s" totalImagesPullingTime="17.026733302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:52:11.807764803 +0000 UTC m=+94.137123677" watchObservedRunningTime="2026-09-06 00:52:11.809234114 +0000 UTC m=+94.138592951"
Sep 06 00:52:12.812121 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:12.812070    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-tnj9f"
Sep 06 00:52:14.882302 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:14.882259    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8676549fbd-q9qzz"]
Sep 06 00:52:14.942567 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:14.942510    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8676549fbd-q9qzz"]
Sep 06 00:52:14.942765 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:14.942632    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:14.950730 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:14.950665    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-oauth-config\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-service-ca\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-trusted-ca-bundle\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-config\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-oauth-serving-cert\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037477    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-serving-cert\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.037630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.037509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndxw5\" (UniqueName: \"kubernetes.io/projected/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-kube-api-access-ndxw5\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.094707 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.094571    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:15.138487 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138400    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-config\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.138487 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-oauth-serving-cert\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.138709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138491    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-serving-cert\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.138709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndxw5\" (UniqueName: \"kubernetes.io/projected/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-kube-api-access-ndxw5\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.138709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138578    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-oauth-config\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.138709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138611    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-service-ca\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.138709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.138648    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-trusted-ca-bundle\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.139254 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.139225    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-config\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.139531 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.139502    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-oauth-serving-cert\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.139824 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.139794    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-service-ca\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.140126 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.140107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-trusted-ca-bundle\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.141527 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.141500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-serving-cert\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.141723 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.141696    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-oauth-config\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.150562 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.150538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndxw5\" (UniqueName: \"kubernetes.io/projected/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-kube-api-access-ndxw5\") pod \"console-8676549fbd-q9qzz\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") " pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.195008 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.194983    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:15.251478 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.251445    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:15.256191 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.256155    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:15.547778 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.547744    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8676549fbd-q9qzz"]
Sep 06 00:52:15.597052 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:52:15.597014    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468ad2aa_1ed6_44ac_a33a_62ebb9231c07.slice/crio-85a73a166ba94733c4842e03fee3a6f5f2ee94c9dbebdbaa470fdf0a73a188fa WatchSource:0}: Error finding container 85a73a166ba94733c4842e03fee3a6f5f2ee94c9dbebdbaa470fdf0a73a188fa: Status 404 returned error can't find the container with id 85a73a166ba94733c4842e03fee3a6f5f2ee94c9dbebdbaa470fdf0a73a188fa
Sep 06 00:52:15.812361 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.812272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fcd9b64d-mblmt" event={"ID":"9a960c83-9f68-4545-ae2d-9bb7b3b473e1","Type":"ContainerStarted","Data":"4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562"}
Sep 06 00:52:15.814194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.814168    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676549fbd-q9qzz" event={"ID":"468ad2aa-1ed6-44ac-a33a-62ebb9231c07","Type":"ContainerStarted","Data":"014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e"}
Sep 06 00:52:15.814302 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.814210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676549fbd-q9qzz" event={"ID":"468ad2aa-1ed6-44ac-a33a-62ebb9231c07","Type":"ContainerStarted","Data":"85a73a166ba94733c4842e03fee3a6f5f2ee94c9dbebdbaa470fdf0a73a188fa"}
Sep 06 00:52:15.831790 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.831735    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77fcd9b64d-mblmt" podStartSLOduration=6.755113735 podStartE2EDuration="10.831714827s" totalImagesPullingTime="4.076601092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:52:15.830308108 +0000 UTC m=+98.159666963" watchObservedRunningTime="2026-09-06 00:52:15.831714827 +0000 UTC m=+98.161073663"
Sep 06 00:52:15.837452 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.837382    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:15.837452 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.837430    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:15.839120 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.838785    2575 patch_prober.go:28] interesting pod/console-77fcd9b64d-mblmt container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused" start-of-body=
Sep 06 00:52:15.839120 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.838833    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-77fcd9b64d-mblmt" podUID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" containerName="console" probeResult="failure" output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused"
Sep 06 00:52:15.850930 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:15.850877    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8676549fbd-q9qzz" podStartSLOduration=1.850829811 podStartE2EDuration="1.850829811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:52:15.848908772 +0000 UTC m=+98.178267620" watchObservedRunningTime="2026-09-06 00:52:15.850829811 +0000 UTC m=+98.180188643"
Sep 06 00:52:19.698114 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:19.698083    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cd6xc"
Sep 06 00:52:24.290248 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:24.290210    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:24.351431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:24.351393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:25.257236 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.257203    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:25.257476 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.257260    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:25.261942 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.261912    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:25.322222 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.322195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:25.377909 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.377841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:25.841225 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.841198    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:25.844885 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.844841    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:25.857577 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.857552    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:52:25.921432 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.921403    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77fcd9b64d-mblmt"]
Sep 06 00:52:25.989382 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:25.989338    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:26.055140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:26.055109    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:26.409262 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:26.409225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:26.470468 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:26.470437    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:27.111088 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:27.111051    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:27.164279 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:27.164249    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:30.177355 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:30.177322    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:30.278621 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:30.278561    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:45.175663 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:45.175623    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:45.230790 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:45.230761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:52:51.877628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:51.877579    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77fcd9b64d-mblmt" podUID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" containerName="console" containerID="cri-o://4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562" gracePeriod=15
Sep 06 00:52:52.152691 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.152637    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:52.203051 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203022    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-config\") pod \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") "
Sep 06 00:52:52.203196 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203060    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-oauth-serving-cert\") pod \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") "
Sep 06 00:52:52.203196 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203084    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-serving-cert\") pod \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") "
Sep 06 00:52:52.203196 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203106    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fms8\" (UniqueName: \"kubernetes.io/projected/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-kube-api-access-6fms8\") pod \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") "
Sep 06 00:52:52.203417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203231    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-oauth-config\") pod \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") "
Sep 06 00:52:52.203468 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203435    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-config" (OuterVolumeSpecName: "console-config") pod "9a960c83-9f68-4545-ae2d-9bb7b3b473e1" (UID: "9a960c83-9f68-4545-ae2d-9bb7b3b473e1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:52:52.203468 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.203445    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a960c83-9f68-4545-ae2d-9bb7b3b473e1" (UID: "9a960c83-9f68-4545-ae2d-9bb7b3b473e1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:52:52.205461 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.205438    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a960c83-9f68-4545-ae2d-9bb7b3b473e1" (UID: "9a960c83-9f68-4545-ae2d-9bb7b3b473e1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:52:52.205461 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.205449    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-kube-api-access-6fms8" (OuterVolumeSpecName: "kube-api-access-6fms8") pod "9a960c83-9f68-4545-ae2d-9bb7b3b473e1" (UID: "9a960c83-9f68-4545-ae2d-9bb7b3b473e1"). InnerVolumeSpecName "kube-api-access-6fms8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:52:52.205625 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.205506    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a960c83-9f68-4545-ae2d-9bb7b3b473e1" (UID: "9a960c83-9f68-4545-ae2d-9bb7b3b473e1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:52:52.304089 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304060    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-service-ca\") pod \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\" (UID: \"9a960c83-9f68-4545-ae2d-9bb7b3b473e1\") "
Sep 06 00:52:52.304209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304195    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-oauth-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:52:52.304209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304207    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:52:52.304276 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304216    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fms8\" (UniqueName: \"kubernetes.io/projected/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-kube-api-access-6fms8\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:52:52.304276 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304225    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-oauth-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:52:52.304276 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304234    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-console-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:52:52.304369 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.304344    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a960c83-9f68-4545-ae2d-9bb7b3b473e1" (UID: "9a960c83-9f68-4545-ae2d-9bb7b3b473e1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:52:52.404579 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.404507    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a960c83-9f68-4545-ae2d-9bb7b3b473e1-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:52:52.956542 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.956511    2575 generic.go:356] "Generic (PLEG): container finished" podID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" containerID="4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562" exitCode=0
Sep 06 00:52:52.956944 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.956558    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77fcd9b64d-mblmt"
Sep 06 00:52:52.956944 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.956599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fcd9b64d-mblmt" event={"ID":"9a960c83-9f68-4545-ae2d-9bb7b3b473e1","Type":"ContainerDied","Data":"4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562"}
Sep 06 00:52:52.956944 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.956648    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77fcd9b64d-mblmt" event={"ID":"9a960c83-9f68-4545-ae2d-9bb7b3b473e1","Type":"ContainerDied","Data":"c059edd35c3adbc87a0e4369774a3e3fa1bdffde82a8779168a089541d47fd19"}
Sep 06 00:52:52.956944 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.956679    2575 scope.go:117] "RemoveContainer" containerID="4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562"
Sep 06 00:52:52.972111 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.972092    2575 scope.go:117] "RemoveContainer" containerID="4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562"
Sep 06 00:52:52.972425 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:52:52.972405    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562\": container with ID starting with 4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562 not found: ID does not exist" containerID="4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562"
Sep 06 00:52:52.972478 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.972434    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562"} err="failed to get container status \"4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562\": rpc error: code = NotFound desc = could not find container \"4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562\": container with ID starting with 4376787e0fc437ed911e0fe6b36a9f888b913a8f449a4e3b086776e1b73dc562 not found: ID does not exist"
Sep 06 00:52:52.980050 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.980026    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77fcd9b64d-mblmt"]
Sep 06 00:52:52.984017 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:52.983995    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77fcd9b64d-mblmt"]
Sep 06 00:52:54.200116 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:52:54.200082    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" path="/var/lib/kubelet/pods/9a960c83-9f68-4545-ae2d-9bb7b3b473e1/volumes"
Sep 06 00:53:04.611077 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:04.611037    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:04.928415 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:04.928326    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:08.160196 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:08.160158    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:09.682219 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.682185    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"]
Sep 06 00:53:09.682596 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.682440    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" containerName="console"
Sep 06 00:53:09.682596 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.682451    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" containerName="console"
Sep 06 00:53:09.682596 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.682493    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a960c83-9f68-4545-ae2d-9bb7b3b473e1" containerName="console"
Sep 06 00:53:09.684555 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.684538    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.687039 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.687019    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 06 00:53:09.687157 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.687140    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-nzq24\""
Sep 06 00:53:09.687463 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.687447    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 06 00:53:09.687544 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.687481    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 06 00:53:09.688000 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.687984    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 06 00:53:09.688084 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.688063    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 06 00:53:09.693421 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.693402    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 06 00:53:09.699780 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.699762    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"]
Sep 06 00:53:09.705713 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-federate-client-tls\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.705817 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-telemeter-client-tls\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.705817 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-serving-certs-ca-bundle\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.705817 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705776    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-metrics-client-ca\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.705817 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705804    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-secret-telemeter-client\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.706020 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8vr6\" (UniqueName: \"kubernetes.io/projected/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-kube-api-access-n8vr6\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.706020 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705875    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.706020 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.705898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806353 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806312    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-telemeter-client-tls\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806353 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-serving-certs-ca-bundle\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-metrics-client-ca\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-secret-telemeter-client\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8vr6\" (UniqueName: \"kubernetes.io/projected/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-kube-api-access-n8vr6\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806731 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806731 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.806731 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.806693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-federate-client-tls\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.807142 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.807115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-serving-certs-ca-bundle\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.807238 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.807166    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-metrics-client-ca\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.808066 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.808041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.808950 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.808933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-telemeter-client-tls\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.809350 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.809327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.809536 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.809522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-federate-client-tls\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.809576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.809546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-secret-telemeter-client\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.814563 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.814546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8vr6\" (UniqueName: \"kubernetes.io/projected/f13c41b5-bc0c-4db1-964c-adc0bcf56ff6-kube-api-access-n8vr6\") pod \"telemeter-client-649b48b4cc-jpvk5\" (UID: \"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6\") " pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:09.993283 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:09.993249    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"
Sep 06 00:53:10.273702 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:10.273609    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-649b48b4cc-jpvk5"]
Sep 06 00:53:10.276534 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:53:10.276508    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13c41b5_bc0c_4db1_964c_adc0bcf56ff6.slice/crio-97a19e4c9dfdfaa51ed85d924ced06f2af9b2cd231634cfc82078764cd86d243 WatchSource:0}: Error finding container 97a19e4c9dfdfaa51ed85d924ced06f2af9b2cd231634cfc82078764cd86d243: Status 404 returned error can't find the container with id 97a19e4c9dfdfaa51ed85d924ced06f2af9b2cd231634cfc82078764cd86d243
Sep 06 00:53:11.024048 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:11.024010    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5" event={"ID":"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6","Type":"ContainerStarted","Data":"97a19e4c9dfdfaa51ed85d924ced06f2af9b2cd231634cfc82078764cd86d243"}
Sep 06 00:53:12.029199 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:12.029163    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5" event={"ID":"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6","Type":"ContainerStarted","Data":"19ecdacfa36169b022b3940ed098a302c5aae40438748687af4cec6575a58952"}
Sep 06 00:53:13.035358 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.035262    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5" event={"ID":"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6","Type":"ContainerStarted","Data":"f3d68aab33b77b381b050eb1b90482be82eefde9da7ebb4cdd45fce0ce65451d"}
Sep 06 00:53:13.035358 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.035325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5" event={"ID":"f13c41b5-bc0c-4db1-964c-adc0bcf56ff6","Type":"ContainerStarted","Data":"44e8669316a430424f9d0627bd7a43d204714f6842da7f3c897bfc71f6583eb3"}
Sep 06 00:53:13.057233 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.057188    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-649b48b4cc-jpvk5" podStartSLOduration=1.6500479430000001 podStartE2EDuration="4.057173293s" totalImagesPullingTime="2.40712535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:53:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:53:13.056819124 +0000 UTC m=+155.386177989" watchObservedRunningTime="2026-09-06 00:53:13.057173293 +0000 UTC m=+155.386532129"
Sep 06 00:53:13.141926 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.141892    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7677767454-jzrgr"]
Sep 06 00:53:13.143916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.143901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.155771 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.155748    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7677767454-jzrgr"]
Sep 06 00:53:13.235395 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-oauth-serving-cert\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.235524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235408    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-config\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.235524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-serving-cert\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.235524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-service-ca\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.235524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235517    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqqfg\" (UniqueName: \"kubernetes.io/projected/0042ae04-f8f8-4b9b-a11e-dbde49028256-kube-api-access-wqqfg\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.235675 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235540    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-oauth-config\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.235675 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.235563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-trusted-ca-bundle\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.274427 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.274401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:13.336551 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336484    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-config\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.336551 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-serving-cert\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.336551 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-service-ca\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.336770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqqfg\" (UniqueName: \"kubernetes.io/projected/0042ae04-f8f8-4b9b-a11e-dbde49028256-kube-api-access-wqqfg\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.336770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-oauth-config\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.336770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-trusted-ca-bundle\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.336770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.336768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-oauth-serving-cert\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.337430 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.337407    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-config\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.337642 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.337418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-oauth-serving-cert\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.337712 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.337573    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-trusted-ca-bundle\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.337712 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.337689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-service-ca\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.339199 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.339181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-oauth-config\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.339248 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.339194    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-serving-cert\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.344793 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.344771    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqqfg\" (UniqueName: \"kubernetes.io/projected/0042ae04-f8f8-4b9b-a11e-dbde49028256-kube-api-access-wqqfg\") pod \"console-7677767454-jzrgr\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") " pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.375208 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.375176    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:13.452420 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.452394    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:13.476280 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.476239    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:13.733627 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.730241    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7677767454-jzrgr"]
Sep 06 00:53:13.733817 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:53:13.733794    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0042ae04_f8f8_4b9b_a11e_dbde49028256.slice/crio-b3ec0c9c7d144074e09fe11de9c8bafd574ff23d0777220e951c88ba405b8208 WatchSource:0}: Error finding container b3ec0c9c7d144074e09fe11de9c8bafd574ff23d0777220e951c88ba405b8208: Status 404 returned error can't find the container with id b3ec0c9c7d144074e09fe11de9c8bafd574ff23d0777220e951c88ba405b8208
Sep 06 00:53:13.940668 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:13.940635    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:14.038877 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:14.038781    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:14.040881 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:14.040841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677767454-jzrgr" event={"ID":"0042ae04-f8f8-4b9b-a11e-dbde49028256","Type":"ContainerStarted","Data":"4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b"}
Sep 06 00:53:14.040997 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:14.040901    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677767454-jzrgr" event={"ID":"0042ae04-f8f8-4b9b-a11e-dbde49028256","Type":"ContainerStarted","Data":"b3ec0c9c7d144074e09fe11de9c8bafd574ff23d0777220e951c88ba405b8208"}
Sep 06 00:53:14.057364 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:14.057323    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7677767454-jzrgr" podStartSLOduration=1.057309758 podStartE2EDuration="1.057309758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:53:14.056221521 +0000 UTC m=+156.385580357" watchObservedRunningTime="2026-09-06 00:53:14.057309758 +0000 UTC m=+156.386668595"
Sep 06 00:53:19.368907 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:19.368842    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:23.453067 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:23.453017    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:23.453460 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:23.453094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:23.458593 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:23.458572    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:24.081792 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:24.081765    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:53:24.127385 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:24.127352    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8676549fbd-q9qzz"]
Sep 06 00:53:24.228146 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:24.228107    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:24.329303 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:24.329268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:24.386891 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:24.386810    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:53:49.151334 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.151279    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8676549fbd-q9qzz" podUID="468ad2aa-1ed6-44ac-a33a-62ebb9231c07" containerName="console" containerID="cri-o://014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e" gracePeriod=15
Sep 06 00:53:49.390867 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.390823    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:53:49.487648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487617    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-serving-cert\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.487793 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487666    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-service-ca\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.487793 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487685    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-oauth-serving-cert\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.487793 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487712    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-trusted-ca-bundle\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.487793 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487737    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndxw5\" (UniqueName: \"kubernetes.io/projected/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-kube-api-access-ndxw5\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.487793 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487787    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-config\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.488056 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.487814    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-oauth-config\") pod \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\" (UID: \"468ad2aa-1ed6-44ac-a33a-62ebb9231c07\") "
Sep 06 00:53:49.488186 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.488144    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-service-ca" (OuterVolumeSpecName: "service-ca") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:53:49.488270 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.488230    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-config" (OuterVolumeSpecName: "console-config") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:53:49.488270 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.488237    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:53:49.488270 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.488252    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:53:49.490013 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.489983    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:53:49.490110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.490020    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:53:49.490161 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.490143    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-kube-api-access-ndxw5" (OuterVolumeSpecName: "kube-api-access-ndxw5") pod "468ad2aa-1ed6-44ac-a33a-62ebb9231c07" (UID: "468ad2aa-1ed6-44ac-a33a-62ebb9231c07"). InnerVolumeSpecName "kube-api-access-ndxw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:53:49.588792 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588757    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:49.588792 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588786    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:49.588792 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588795    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-oauth-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:49.589013 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588803    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-trusted-ca-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:49.589013 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588812    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndxw5\" (UniqueName: \"kubernetes.io/projected/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-kube-api-access-ndxw5\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:49.589013 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588821    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:49.589013 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:49.588829    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/468ad2aa-1ed6-44ac-a33a-62ebb9231c07-console-oauth-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:53:50.168984 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.168946    2575 generic.go:356] "Generic (PLEG): container finished" podID="468ad2aa-1ed6-44ac-a33a-62ebb9231c07" containerID="014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e" exitCode=0
Sep 06 00:53:50.169381 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.168989    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676549fbd-q9qzz" event={"ID":"468ad2aa-1ed6-44ac-a33a-62ebb9231c07","Type":"ContainerDied","Data":"014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e"}
Sep 06 00:53:50.169381 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.168998    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676549fbd-q9qzz"
Sep 06 00:53:50.169381 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.169040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676549fbd-q9qzz" event={"ID":"468ad2aa-1ed6-44ac-a33a-62ebb9231c07","Type":"ContainerDied","Data":"85a73a166ba94733c4842e03fee3a6f5f2ee94c9dbebdbaa470fdf0a73a188fa"}
Sep 06 00:53:50.169381 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.169067    2575 scope.go:117] "RemoveContainer" containerID="014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e"
Sep 06 00:53:50.178994 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.178974    2575 scope.go:117] "RemoveContainer" containerID="014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e"
Sep 06 00:53:50.179244 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:53:50.179227    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e\": container with ID starting with 014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e not found: ID does not exist" containerID="014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e"
Sep 06 00:53:50.179301 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.179252    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e"} err="failed to get container status \"014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e\": rpc error: code = NotFound desc = could not find container \"014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e\": container with ID starting with 014f3beb92eb9884abc4ee9d40cb0459a569fe4da6f1d8dc2297328885358d1e not found: ID does not exist"
Sep 06 00:53:50.188678 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.188658    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8676549fbd-q9qzz"]
Sep 06 00:53:50.192429 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.192410    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8676549fbd-q9qzz"]
Sep 06 00:53:50.199708 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:50.199689    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468ad2aa-1ed6-44ac-a33a-62ebb9231c07" path="/var/lib/kubelet/pods/468ad2aa-1ed6-44ac-a33a-62ebb9231c07/volumes"
Sep 06 00:53:58.824495 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:53:58.824451    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:19.811417 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:19.811363    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:19.910722 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:19.910686    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:31.852732 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:31.852693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:31.907357 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:31.907319    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:45.188337 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:45.188290    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:45.289639 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:45.289606    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:47.678111 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.678068    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744dcf646b-xstsg"]
Sep 06 00:54:47.678490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.678357    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468ad2aa-1ed6-44ac-a33a-62ebb9231c07" containerName="console"
Sep 06 00:54:47.678490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.678368    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="468ad2aa-1ed6-44ac-a33a-62ebb9231c07" containerName="console"
Sep 06 00:54:47.678490 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.678424    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="468ad2aa-1ed6-44ac-a33a-62ebb9231c07" containerName="console"
Sep 06 00:54:47.682400 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.682379    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.692240 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.692219    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744dcf646b-xstsg"]
Sep 06 00:54:47.709241 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rzfv\" (UniqueName: \"kubernetes.io/projected/be1f5342-72d6-4ed7-89ac-db40f3035977-kube-api-access-8rzfv\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.709241 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-oauth-serving-cert\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.709373 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-trusted-ca-bundle\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.709373 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-oauth-config\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.709373 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-service-ca\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.709373 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-console-config\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.709517 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.709421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-serving-cert\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.734172 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.734131    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:47.789871 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.789826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:47.809810 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-trusted-ca-bundle\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.809908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-oauth-config\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.809908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-service-ca\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.809908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-console-config\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.809908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-serving-cert\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.810117 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rzfv\" (UniqueName: \"kubernetes.io/projected/be1f5342-72d6-4ed7-89ac-db40f3035977-kube-api-access-8rzfv\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.810117 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.809987    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-oauth-serving-cert\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.810637 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.810608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-service-ca\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.810637 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.810631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-console-config\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.810960 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.810704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-oauth-serving-cert\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.810960 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.810759    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-trusted-ca-bundle\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.812502 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.812482    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-oauth-config\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.812611 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.812592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-serving-cert\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.820038 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.820019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rzfv\" (UniqueName: \"kubernetes.io/projected/be1f5342-72d6-4ed7-89ac-db40f3035977-kube-api-access-8rzfv\") pod \"console-744dcf646b-xstsg\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") " pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:47.990948 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:47.990918    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:48.264198 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:48.264128    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744dcf646b-xstsg"]
Sep 06 00:54:48.267483 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:54:48.267456    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1f5342_72d6_4ed7_89ac_db40f3035977.slice/crio-8bf01357e08411d27d0a40955bffbcef232eeb48fbf5b8810f403bc14a98174f WatchSource:0}: Error finding container 8bf01357e08411d27d0a40955bffbcef232eeb48fbf5b8810f403bc14a98174f: Status 404 returned error can't find the container with id 8bf01357e08411d27d0a40955bffbcef232eeb48fbf5b8810f403bc14a98174f
Sep 06 00:54:48.367001 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:48.366964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744dcf646b-xstsg" event={"ID":"be1f5342-72d6-4ed7-89ac-db40f3035977","Type":"ContainerStarted","Data":"a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283"}
Sep 06 00:54:48.367174 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:48.367056    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744dcf646b-xstsg" event={"ID":"be1f5342-72d6-4ed7-89ac-db40f3035977","Type":"ContainerStarted","Data":"8bf01357e08411d27d0a40955bffbcef232eeb48fbf5b8810f403bc14a98174f"}
Sep 06 00:54:48.383508 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:48.383450    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744dcf646b-xstsg" podStartSLOduration=1.383430415 podStartE2EDuration="1.383430415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:54:48.382645656 +0000 UTC m=+250.712004504" watchObservedRunningTime="2026-09-06 00:54:48.383430415 +0000 UTC m=+250.712789277"
Sep 06 00:54:57.991426 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:57.991389    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:57.991843 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:57.991452    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:57.996388 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:57.996367    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:58.409014 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:58.408930    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:54:58.459839 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:58.459794    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7677767454-jzrgr"]
Sep 06 00:54:58.566629 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:58.566595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:58.670053 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:58.669968    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:54:58.726068 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:54:58.726029    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:55:23.480062 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.480000    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7677767454-jzrgr" podUID="0042ae04-f8f8-4b9b-a11e-dbde49028256" containerName="console" containerID="cri-o://4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b" gracePeriod=15
Sep 06 00:55:23.722943 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.722920    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:55:23.786628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786565    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqqfg\" (UniqueName: \"kubernetes.io/projected/0042ae04-f8f8-4b9b-a11e-dbde49028256-kube-api-access-wqqfg\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.786628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786623    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-trusted-ca-bundle\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.786815 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786641    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-config\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.786815 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786670    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-oauth-config\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.786815 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786702    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-service-ca\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.786815 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786731    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-serving-cert\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.786815 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.786763    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-oauth-serving-cert\") pod \"0042ae04-f8f8-4b9b-a11e-dbde49028256\" (UID: \"0042ae04-f8f8-4b9b-a11e-dbde49028256\") "
Sep 06 00:55:23.787133 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.787103    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:55:23.787189 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.787165    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-config" (OuterVolumeSpecName: "console-config") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:55:23.787238 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.787187    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:55:23.787238 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.787194    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-service-ca" (OuterVolumeSpecName: "service-ca") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:55:23.788837 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.788812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:55:23.789005 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.788983    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0042ae04-f8f8-4b9b-a11e-dbde49028256-kube-api-access-wqqfg" (OuterVolumeSpecName: "kube-api-access-wqqfg") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "kube-api-access-wqqfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:55:23.789062 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.789001    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0042ae04-f8f8-4b9b-a11e-dbde49028256" (UID: "0042ae04-f8f8-4b9b-a11e-dbde49028256"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:55:23.887916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887886    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:23.887916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887910    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:23.887916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887920    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-oauth-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:23.888106 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887928    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqqfg\" (UniqueName: \"kubernetes.io/projected/0042ae04-f8f8-4b9b-a11e-dbde49028256-kube-api-access-wqqfg\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:23.888106 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887937    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-trusted-ca-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:23.888106 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887946    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:23.888106 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:23.887955    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0042ae04-f8f8-4b9b-a11e-dbde49028256-console-oauth-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:55:24.499772 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.499688    2575 generic.go:356] "Generic (PLEG): container finished" podID="0042ae04-f8f8-4b9b-a11e-dbde49028256" containerID="4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b" exitCode=0
Sep 06 00:55:24.499772 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.499738    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677767454-jzrgr"
Sep 06 00:55:24.500195 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.499739    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677767454-jzrgr" event={"ID":"0042ae04-f8f8-4b9b-a11e-dbde49028256","Type":"ContainerDied","Data":"4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b"}
Sep 06 00:55:24.500195 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.499913    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677767454-jzrgr" event={"ID":"0042ae04-f8f8-4b9b-a11e-dbde49028256","Type":"ContainerDied","Data":"b3ec0c9c7d144074e09fe11de9c8bafd574ff23d0777220e951c88ba405b8208"}
Sep 06 00:55:24.500195 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.499947    2575 scope.go:117] "RemoveContainer" containerID="4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b"
Sep 06 00:55:24.507837 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.507822    2575 scope.go:117] "RemoveContainer" containerID="4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b"
Sep 06 00:55:24.508166 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:55:24.508149    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b\": container with ID starting with 4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b not found: ID does not exist" containerID="4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b"
Sep 06 00:55:24.508209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.508175    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b"} err="failed to get container status \"4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b\": rpc error: code = NotFound desc = could not find container \"4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b\": container with ID starting with 4a48d2110ada1d3ae7119c505cdbd1d132ef89454e9e436e64733913078cbe9b not found: ID does not exist"
Sep 06 00:55:24.518887 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.518866    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7677767454-jzrgr"]
Sep 06 00:55:24.521311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:24.521292    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7677767454-jzrgr"]
Sep 06 00:55:26.200173 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:26.200139    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0042ae04-f8f8-4b9b-a11e-dbde49028256" path="/var/lib/kubelet/pods/0042ae04-f8f8-4b9b-a11e-dbde49028256/volumes"
Sep 06 00:55:38.109875 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:38.109824    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:55:38.110631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:38.110612    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 00:55:38.116243 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:38.116225    2575 kubelet.go:1648] "Image garbage collection succeeded"
Sep 06 00:55:43.664931 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:43.664892    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:55:44.374231 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.374197    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"]
Sep 06 00:55:44.374464 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.374453    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0042ae04-f8f8-4b9b-a11e-dbde49028256" containerName="console"
Sep 06 00:55:44.374510 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.374466    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0042ae04-f8f8-4b9b-a11e-dbde49028256" containerName="console"
Sep 06 00:55:44.374544 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.374517    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0042ae04-f8f8-4b9b-a11e-dbde49028256" containerName="console"
Sep 06 00:55:44.377350 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.377329    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.379866 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.379820    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 00:55:44.379988 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.379828    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 00:55:44.379988 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.379929    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l54wx\""
Sep 06 00:55:44.388752 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.388730    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"]
Sep 06 00:55:44.541589 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.541546    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pt7v\" (UniqueName: \"kubernetes.io/projected/3ede2df0-dece-4526-8479-705b97acc710-kube-api-access-6pt7v\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.541769 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.541622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.541769 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.541660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.642874 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.642784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pt7v\" (UniqueName: \"kubernetes.io/projected/3ede2df0-dece-4526-8479-705b97acc710-kube-api-access-6pt7v\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.642874 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.642830    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.642874 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.642871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.643217 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.643197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.643290 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.643235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.652665 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.652643    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pt7v\" (UniqueName: \"kubernetes.io/projected/3ede2df0-dece-4526-8479-705b97acc710-kube-api-access-6pt7v\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.686423 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.686407    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:55:44.953785 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.953761    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"]
Sep 06 00:55:44.955714 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:55:44.955686    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ede2df0_dece_4526_8479_705b97acc710.slice/crio-657d13aa9af720602976b6395aa5eec95f1e2b7fc1ad107185e8fb945488908f WatchSource:0}: Error finding container 657d13aa9af720602976b6395aa5eec95f1e2b7fc1ad107185e8fb945488908f: Status 404 returned error can't find the container with id 657d13aa9af720602976b6395aa5eec95f1e2b7fc1ad107185e8fb945488908f
Sep 06 00:55:44.957562 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:44.957544    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 00:55:45.043079 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:45.043044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:55:45.576650 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:45.576608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w" event={"ID":"3ede2df0-dece-4526-8479-705b97acc710","Type":"ContainerStarted","Data":"657d13aa9af720602976b6395aa5eec95f1e2b7fc1ad107185e8fb945488908f"}
Sep 06 00:55:50.596381 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:50.596354    2575 generic.go:356] "Generic (PLEG): container finished" podID="3ede2df0-dece-4526-8479-705b97acc710" containerID="0281b91d790b9d16e4b7e468fb8d1d0dcff7a364a20d305a89549423dd66327c" exitCode=0
Sep 06 00:55:50.596722 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:50.596444    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w" event={"ID":"3ede2df0-dece-4526-8479-705b97acc710","Type":"ContainerDied","Data":"0281b91d790b9d16e4b7e468fb8d1d0dcff7a364a20d305a89549423dd66327c"}
Sep 06 00:55:53.608970 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:53.608930    2575 generic.go:356] "Generic (PLEG): container finished" podID="3ede2df0-dece-4526-8479-705b97acc710" containerID="9f1d617dd47b334fead6462bb47596280df8cd025e3dab66f7a789d8475ff251" exitCode=0
Sep 06 00:55:53.609363 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:55:53.609013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w" event={"ID":"3ede2df0-dece-4526-8479-705b97acc710","Type":"ContainerDied","Data":"9f1d617dd47b334fead6462bb47596280df8cd025e3dab66f7a789d8475ff251"}
Sep 06 00:56:00.639019 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:00.638939    2575 generic.go:356] "Generic (PLEG): container finished" podID="3ede2df0-dece-4526-8479-705b97acc710" containerID="6fdac8c386ee724646b32c39b4784f6935d0713333da5970bdb69f6401bce967" exitCode=0
Sep 06 00:56:00.639019 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:00.638991    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w" event={"ID":"3ede2df0-dece-4526-8479-705b97acc710","Type":"ContainerDied","Data":"6fdac8c386ee724646b32c39b4784f6935d0713333da5970bdb69f6401bce967"}
Sep 06 00:56:01.769743 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.769719    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:56:01.867283 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.867242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-bundle\") pod \"3ede2df0-dece-4526-8479-705b97acc710\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") "
Sep 06 00:56:01.867283 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.867286    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pt7v\" (UniqueName: \"kubernetes.io/projected/3ede2df0-dece-4526-8479-705b97acc710-kube-api-access-6pt7v\") pod \"3ede2df0-dece-4526-8479-705b97acc710\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") "
Sep 06 00:56:01.867528 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.867321    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-util\") pod \"3ede2df0-dece-4526-8479-705b97acc710\" (UID: \"3ede2df0-dece-4526-8479-705b97acc710\") "
Sep 06 00:56:01.867992 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.867966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-bundle" (OuterVolumeSpecName: "bundle") pod "3ede2df0-dece-4526-8479-705b97acc710" (UID: "3ede2df0-dece-4526-8479-705b97acc710"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:01.869618 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.869597    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ede2df0-dece-4526-8479-705b97acc710-kube-api-access-6pt7v" (OuterVolumeSpecName: "kube-api-access-6pt7v") pod "3ede2df0-dece-4526-8479-705b97acc710" (UID: "3ede2df0-dece-4526-8479-705b97acc710"). InnerVolumeSpecName "kube-api-access-6pt7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:56:01.873345 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.873317    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-util" (OuterVolumeSpecName: "util") pod "3ede2df0-dece-4526-8479-705b97acc710" (UID: "3ede2df0-dece-4526-8479-705b97acc710"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:01.968119 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.968086    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pt7v\" (UniqueName: \"kubernetes.io/projected/3ede2df0-dece-4526-8479-705b97acc710-kube-api-access-6pt7v\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:01.968119 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.968117    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:01.968284 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:01.968127    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ede2df0-dece-4526-8479-705b97acc710-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:02.647521 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:02.647487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w" event={"ID":"3ede2df0-dece-4526-8479-705b97acc710","Type":"ContainerDied","Data":"657d13aa9af720602976b6395aa5eec95f1e2b7fc1ad107185e8fb945488908f"}
Sep 06 00:56:02.647706 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:02.647540    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="657d13aa9af720602976b6395aa5eec95f1e2b7fc1ad107185e8fb945488908f"
Sep 06 00:56:02.647706 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:02.647494    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540psp8w"
Sep 06 00:56:05.576905 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:05.576873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:06.934203 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:06.934166    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:07.566999 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.566972    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"]
Sep 06 00:56:07.567251 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567238    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="pull"
Sep 06 00:56:07.567295 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567253    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="pull"
Sep 06 00:56:07.567295 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567278    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="extract"
Sep 06 00:56:07.567295 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567284    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="extract"
Sep 06 00:56:07.567295 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567289    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="util"
Sep 06 00:56:07.567295 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567294    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="util"
Sep 06 00:56:07.567436 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.567340    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ede2df0-dece-4526-8479-705b97acc710" containerName="extract"
Sep 06 00:56:07.608566 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.608538    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"]
Sep 06 00:56:07.608701 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.608637    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.611240 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.611213    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mvx5j\""
Sep 06 00:56:07.611378 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.611220    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 06 00:56:07.611378 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.611220    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 06 00:56:07.710412 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.710380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/942e15c7-007a-463d-a9bf-f835d21df84f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gdkqh\" (UID: \"942e15c7-007a-463d-a9bf-f835d21df84f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.710552 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.710414    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzsss\" (UniqueName: \"kubernetes.io/projected/942e15c7-007a-463d-a9bf-f835d21df84f-kube-api-access-wzsss\") pod \"cert-manager-operator-controller-manager-55c444bf86-gdkqh\" (UID: \"942e15c7-007a-463d-a9bf-f835d21df84f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.811452 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.811425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/942e15c7-007a-463d-a9bf-f835d21df84f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gdkqh\" (UID: \"942e15c7-007a-463d-a9bf-f835d21df84f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.811675 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.811655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzsss\" (UniqueName: \"kubernetes.io/projected/942e15c7-007a-463d-a9bf-f835d21df84f-kube-api-access-wzsss\") pod \"cert-manager-operator-controller-manager-55c444bf86-gdkqh\" (UID: \"942e15c7-007a-463d-a9bf-f835d21df84f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.812228 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.812202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/942e15c7-007a-463d-a9bf-f835d21df84f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gdkqh\" (UID: \"942e15c7-007a-463d-a9bf-f835d21df84f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.825139 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.825073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzsss\" (UniqueName: \"kubernetes.io/projected/942e15c7-007a-463d-a9bf-f835d21df84f-kube-api-access-wzsss\") pod \"cert-manager-operator-controller-manager-55c444bf86-gdkqh\" (UID: \"942e15c7-007a-463d-a9bf-f835d21df84f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:07.917239 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:07.917202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"
Sep 06 00:56:08.215148 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:08.215120    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh"]
Sep 06 00:56:08.218252 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:08.218223    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942e15c7_007a_463d_a9bf_f835d21df84f.slice/crio-a752ca7bd6b8ef4bd950234f2c774812cbaa26b7b90b044bdba1b512ad4a33b7 WatchSource:0}: Error finding container a752ca7bd6b8ef4bd950234f2c774812cbaa26b7b90b044bdba1b512ad4a33b7: Status 404 returned error can't find the container with id a752ca7bd6b8ef4bd950234f2c774812cbaa26b7b90b044bdba1b512ad4a33b7
Sep 06 00:56:08.669724 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:08.669645    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh" event={"ID":"942e15c7-007a-463d-a9bf-f835d21df84f","Type":"ContainerStarted","Data":"a752ca7bd6b8ef4bd950234f2c774812cbaa26b7b90b044bdba1b512ad4a33b7"}
Sep 06 00:56:09.182791 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:09.182728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:09.286380 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:09.286320    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:10.679715 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:10.679681    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh" event={"ID":"942e15c7-007a-463d-a9bf-f835d21df84f","Type":"ContainerStarted","Data":"df8413bf49c29067aebc4d9a4e6fe71238803750ea4dddce19f01dbcc8670db1"}
Sep 06 00:56:10.699366 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:10.699324    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gdkqh" podStartSLOduration=1.779589343 podStartE2EDuration="3.699309462s" totalImagesPullingTime="1.919720119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:56:10.698952187 +0000 UTC m=+333.028311033" watchObservedRunningTime="2026-09-06 00:56:10.699309462 +0000 UTC m=+333.028668306"
Sep 06 00:56:10.989495 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:10.989448    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:11.099961 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.099927    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:11.208663 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.208630    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nklxv"]
Sep 06 00:56:11.220340 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.220320    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.223976 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.223956    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 06 00:56:11.224230 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.224214    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 06 00:56:11.227903 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.227880    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nklxv"]
Sep 06 00:56:11.233336 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.233310    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:11.239431 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.239407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blsf2\" (UniqueName: \"kubernetes.io/projected/d20b04f6-cff7-4331-94a2-35ea41fa6636-kube-api-access-blsf2\") pod \"cert-manager-webhook-6486ffd997-nklxv\" (UID: \"d20b04f6-cff7-4331-94a2-35ea41fa6636\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.239527 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.239440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d20b04f6-cff7-4331-94a2-35ea41fa6636-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nklxv\" (UID: \"d20b04f6-cff7-4331-94a2-35ea41fa6636\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.340724 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.340688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blsf2\" (UniqueName: \"kubernetes.io/projected/d20b04f6-cff7-4331-94a2-35ea41fa6636-kube-api-access-blsf2\") pod \"cert-manager-webhook-6486ffd997-nklxv\" (UID: \"d20b04f6-cff7-4331-94a2-35ea41fa6636\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.340724 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.340724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d20b04f6-cff7-4331-94a2-35ea41fa6636-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nklxv\" (UID: \"d20b04f6-cff7-4331-94a2-35ea41fa6636\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.351007 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.350977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:11.360275 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.360247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blsf2\" (UniqueName: \"kubernetes.io/projected/d20b04f6-cff7-4331-94a2-35ea41fa6636-kube-api-access-blsf2\") pod \"cert-manager-webhook-6486ffd997-nklxv\" (UID: \"d20b04f6-cff7-4331-94a2-35ea41fa6636\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.360804 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.360784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d20b04f6-cff7-4331-94a2-35ea41fa6636-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nklxv\" (UID: \"d20b04f6-cff7-4331-94a2-35ea41fa6636\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.529836 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.529748    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:11.801672 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:11.801562    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nklxv"]
Sep 06 00:56:11.805431 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:11.805400    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd20b04f6_cff7_4331_94a2_35ea41fa6636.slice/crio-de8a760f1e48c4807ba3198e30feb3c47a05d6ced586e731bb29fd3b04764f77 WatchSource:0}: Error finding container de8a760f1e48c4807ba3198e30feb3c47a05d6ced586e731bb29fd3b04764f77: Status 404 returned error can't find the container with id de8a760f1e48c4807ba3198e30feb3c47a05d6ced586e731bb29fd3b04764f77
Sep 06 00:56:12.015634 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.015593    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"]
Sep 06 00:56:12.018802 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.018781    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.021154 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.021134    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 00:56:12.021252 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.021215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l54wx\""
Sep 06 00:56:12.021303 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.021269    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 00:56:12.028273 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.028253    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"]
Sep 06 00:56:12.048199 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.048172    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.048309 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.048207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.048309 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.048244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdz8\" (UniqueName: \"kubernetes.io/projected/84104dd3-3414-42a6-b42f-8ec732a3187a-kube-api-access-cwdz8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.149301 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.149217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.149301 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.149276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.149501 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.149341    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdz8\" (UniqueName: \"kubernetes.io/projected/84104dd3-3414-42a6-b42f-8ec732a3187a-kube-api-access-cwdz8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.149632 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.149615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.149676 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.149657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.162009 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.160380    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdz8\" (UniqueName: \"kubernetes.io/projected/84104dd3-3414-42a6-b42f-8ec732a3187a-kube-api-access-cwdz8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.329106 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.329067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:12.601309 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.601254    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"]
Sep 06 00:56:12.603622 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:12.603598    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84104dd3_3414_42a6_b42f_8ec732a3187a.slice/crio-548d663fd4965f1006e9978ae74b711907c434198d64bc0d0306b6de092ef2de WatchSource:0}: Error finding container 548d663fd4965f1006e9978ae74b711907c434198d64bc0d0306b6de092ef2de: Status 404 returned error can't find the container with id 548d663fd4965f1006e9978ae74b711907c434198d64bc0d0306b6de092ef2de
Sep 06 00:56:12.688583 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.688550    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74" event={"ID":"84104dd3-3414-42a6-b42f-8ec732a3187a","Type":"ContainerStarted","Data":"7aaafbe9f8e7e9224f25fd164d7f2ef8dc2555d2be948bb5559fa6ba5470a25e"}
Sep 06 00:56:12.688744 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.688611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74" event={"ID":"84104dd3-3414-42a6-b42f-8ec732a3187a","Type":"ContainerStarted","Data":"548d663fd4965f1006e9978ae74b711907c434198d64bc0d0306b6de092ef2de"}
Sep 06 00:56:12.689948 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:12.689924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv" event={"ID":"d20b04f6-cff7-4331-94a2-35ea41fa6636","Type":"ContainerStarted","Data":"de8a760f1e48c4807ba3198e30feb3c47a05d6ced586e731bb29fd3b04764f77"}
Sep 06 00:56:13.200801 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:13.200764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:13.589325 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:13.589238    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:13.695835 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:13.695802    2575 generic.go:356] "Generic (PLEG): container finished" podID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerID="7aaafbe9f8e7e9224f25fd164d7f2ef8dc2555d2be948bb5559fa6ba5470a25e" exitCode=0
Sep 06 00:56:13.696054 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:13.695885    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74" event={"ID":"84104dd3-3414-42a6-b42f-8ec732a3187a","Type":"ContainerDied","Data":"7aaafbe9f8e7e9224f25fd164d7f2ef8dc2555d2be948bb5559fa6ba5470a25e"}
Sep 06 00:56:16.177544 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:16.177508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:16.278724 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:16.278666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:17.720375 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:17.720341    2575 generic.go:356] "Generic (PLEG): container finished" podID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerID="6f4d9e893672aa5128ef7cf8b778e11b5108852f5c59cac5623bbd9ce322d829" exitCode=0
Sep 06 00:56:17.720776 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:17.720436    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74" event={"ID":"84104dd3-3414-42a6-b42f-8ec732a3187a","Type":"ContainerDied","Data":"6f4d9e893672aa5128ef7cf8b778e11b5108852f5c59cac5623bbd9ce322d829"}
Sep 06 00:56:17.722291 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:17.722266    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv" event={"ID":"d20b04f6-cff7-4331-94a2-35ea41fa6636","Type":"ContainerStarted","Data":"95ff81aa485ece2d426a32da8280ced3ae03631f944ae7148ca0632b52b7b7e4"}
Sep 06 00:56:17.722375 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:17.722309    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:17.754726 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:17.754681    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv" podStartSLOduration=1.913034749 podStartE2EDuration="6.754665508s" totalImagesPullingTime="4.841630759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:56:17.753799551 +0000 UTC m=+340.083158387" watchObservedRunningTime="2026-09-06 00:56:17.754665508 +0000 UTC m=+340.084024343"
Sep 06 00:56:18.728409 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:18.728373    2575 generic.go:356] "Generic (PLEG): container finished" podID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerID="02c7d65a4a858103b254e175a963b342713bce94305ec3ca0444655e4b12b0fe" exitCode=0
Sep 06 00:56:18.728752 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:18.728455    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74" event={"ID":"84104dd3-3414-42a6-b42f-8ec732a3187a","Type":"ContainerDied","Data":"02c7d65a4a858103b254e175a963b342713bce94305ec3ca0444655e4b12b0fe"}
Sep 06 00:56:19.853245 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.853223    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:19.917723 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.917692    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-util\") pod \"84104dd3-3414-42a6-b42f-8ec732a3187a\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") "
Sep 06 00:56:19.917893 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.917742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-bundle\") pod \"84104dd3-3414-42a6-b42f-8ec732a3187a\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") "
Sep 06 00:56:19.917893 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.917786    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwdz8\" (UniqueName: \"kubernetes.io/projected/84104dd3-3414-42a6-b42f-8ec732a3187a-kube-api-access-cwdz8\") pod \"84104dd3-3414-42a6-b42f-8ec732a3187a\" (UID: \"84104dd3-3414-42a6-b42f-8ec732a3187a\") "
Sep 06 00:56:19.918129 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.918103    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-bundle" (OuterVolumeSpecName: "bundle") pod "84104dd3-3414-42a6-b42f-8ec732a3187a" (UID: "84104dd3-3414-42a6-b42f-8ec732a3187a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:19.920110 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.920079    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84104dd3-3414-42a6-b42f-8ec732a3187a-kube-api-access-cwdz8" (OuterVolumeSpecName: "kube-api-access-cwdz8") pod "84104dd3-3414-42a6-b42f-8ec732a3187a" (UID: "84104dd3-3414-42a6-b42f-8ec732a3187a"). InnerVolumeSpecName "kube-api-access-cwdz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:56:19.922510 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:19.922475    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-util" (OuterVolumeSpecName: "util") pod "84104dd3-3414-42a6-b42f-8ec732a3187a" (UID: "84104dd3-3414-42a6-b42f-8ec732a3187a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:20.019094 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:20.019025    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwdz8\" (UniqueName: \"kubernetes.io/projected/84104dd3-3414-42a6-b42f-8ec732a3187a-kube-api-access-cwdz8\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:20.019094 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:20.019049    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:20.019094 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:20.019058    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84104dd3-3414-42a6-b42f-8ec732a3187a-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:20.739127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:20.739085    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74" event={"ID":"84104dd3-3414-42a6-b42f-8ec732a3187a","Type":"ContainerDied","Data":"548d663fd4965f1006e9978ae74b711907c434198d64bc0d0306b6de092ef2de"}
Sep 06 00:56:20.739127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:20.739115    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8c74"
Sep 06 00:56:20.739313 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:20.739141    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="548d663fd4965f1006e9978ae74b711907c434198d64bc0d0306b6de092ef2de"
Sep 06 00:56:21.187267 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:21.187186    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:22.238488 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238453    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h8dxw"]
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238699    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="extract"
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238710    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="extract"
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238726    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="pull"
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238731    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="pull"
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238744    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="util"
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238749    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="util"
Sep 06 00:56:22.238883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.238791    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="84104dd3-3414-42a6-b42f-8ec732a3187a" containerName="extract"
Sep 06 00:56:22.240660 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.240646    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.243446 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.243415    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-t8jmz\""
Sep 06 00:56:22.249469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.249450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h8dxw"]
Sep 06 00:56:22.335371 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.335342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7v8d\" (UniqueName: \"kubernetes.io/projected/4867ad23-4605-4d37-84fb-6106144a2c09-kube-api-access-p7v8d\") pod \"cert-manager-55b4d954d5-h8dxw\" (UID: \"4867ad23-4605-4d37-84fb-6106144a2c09\") " pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.335506 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.335402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4867ad23-4605-4d37-84fb-6106144a2c09-bound-sa-token\") pod \"cert-manager-55b4d954d5-h8dxw\" (UID: \"4867ad23-4605-4d37-84fb-6106144a2c09\") " pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.435937 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.435884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7v8d\" (UniqueName: \"kubernetes.io/projected/4867ad23-4605-4d37-84fb-6106144a2c09-kube-api-access-p7v8d\") pod \"cert-manager-55b4d954d5-h8dxw\" (UID: \"4867ad23-4605-4d37-84fb-6106144a2c09\") " pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.436061 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.435970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4867ad23-4605-4d37-84fb-6106144a2c09-bound-sa-token\") pod \"cert-manager-55b4d954d5-h8dxw\" (UID: \"4867ad23-4605-4d37-84fb-6106144a2c09\") " pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.445415 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.445393    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7v8d\" (UniqueName: \"kubernetes.io/projected/4867ad23-4605-4d37-84fb-6106144a2c09-kube-api-access-p7v8d\") pod \"cert-manager-55b4d954d5-h8dxw\" (UID: \"4867ad23-4605-4d37-84fb-6106144a2c09\") " pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.445513 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.445417    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4867ad23-4605-4d37-84fb-6106144a2c09-bound-sa-token\") pod \"cert-manager-55b4d954d5-h8dxw\" (UID: \"4867ad23-4605-4d37-84fb-6106144a2c09\") " pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.549709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.549635    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h8dxw"
Sep 06 00:56:22.815869 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:22.815830    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h8dxw"]
Sep 06 00:56:22.818256 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:22.818231    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4867ad23_4605_4d37_84fb_6106144a2c09.slice/crio-8b6009a3f4fefacd4723c60b0fb5a7be1c587109ebf7de4a27fa4f37ad59ef9f WatchSource:0}: Error finding container 8b6009a3f4fefacd4723c60b0fb5a7be1c587109ebf7de4a27fa4f37ad59ef9f: Status 404 returned error can't find the container with id 8b6009a3f4fefacd4723c60b0fb5a7be1c587109ebf7de4a27fa4f37ad59ef9f
Sep 06 00:56:23.504342 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:23.504311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:23.575003 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:23.574975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:23.731180 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:23.731153    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nklxv"
Sep 06 00:56:23.752954 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:23.752922    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h8dxw" event={"ID":"4867ad23-4605-4d37-84fb-6106144a2c09","Type":"ContainerStarted","Data":"d644b945efd9fa382be1bb253eb1fdc95e89dc189bfda0c7fdca9c437a97cf91"}
Sep 06 00:56:23.752954 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:23.752956    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h8dxw" event={"ID":"4867ad23-4605-4d37-84fb-6106144a2c09","Type":"ContainerStarted","Data":"8b6009a3f4fefacd4723c60b0fb5a7be1c587109ebf7de4a27fa4f37ad59ef9f"}
Sep 06 00:56:23.771675 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:23.771578    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-h8dxw" podStartSLOduration=1.7715565450000001 podStartE2EDuration="1.771556545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:56:23.770823427 +0000 UTC m=+346.100182262" watchObservedRunningTime="2026-09-06 00:56:23.771556545 +0000 UTC m=+346.100915382"
Sep 06 00:56:24.614088 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:24.614055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:24.714171 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:24.714144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:26.293763 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.293732    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"]
Sep 06 00:56:26.295827 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.295811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.298225 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.298197    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 06 00:56:26.298905 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.298888    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lfss9\""
Sep 06 00:56:26.298997 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.298892    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 06 00:56:26.309732 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.309711    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"]
Sep 06 00:56:26.365316 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.365275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdmqz\" (UniqueName: \"kubernetes.io/projected/a7cfb069-044d-4179-9a64-6f3466349795-kube-api-access-wdmqz\") pod \"openshift-lws-operator-fd8ccff4c-trx9n\" (UID: \"a7cfb069-044d-4179-9a64-6f3466349795\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.365484 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.365325    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7cfb069-044d-4179-9a64-6f3466349795-tmp\") pod \"openshift-lws-operator-fd8ccff4c-trx9n\" (UID: \"a7cfb069-044d-4179-9a64-6f3466349795\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.466188 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.466154    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmqz\" (UniqueName: \"kubernetes.io/projected/a7cfb069-044d-4179-9a64-6f3466349795-kube-api-access-wdmqz\") pod \"openshift-lws-operator-fd8ccff4c-trx9n\" (UID: \"a7cfb069-044d-4179-9a64-6f3466349795\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.466288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.466199    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7cfb069-044d-4179-9a64-6f3466349795-tmp\") pod \"openshift-lws-operator-fd8ccff4c-trx9n\" (UID: \"a7cfb069-044d-4179-9a64-6f3466349795\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.466525 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.466511    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7cfb069-044d-4179-9a64-6f3466349795-tmp\") pod \"openshift-lws-operator-fd8ccff4c-trx9n\" (UID: \"a7cfb069-044d-4179-9a64-6f3466349795\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.475728 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.475704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdmqz\" (UniqueName: \"kubernetes.io/projected/a7cfb069-044d-4179-9a64-6f3466349795-kube-api-access-wdmqz\") pod \"openshift-lws-operator-fd8ccff4c-trx9n\" (UID: \"a7cfb069-044d-4179-9a64-6f3466349795\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.605247 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.605160    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"
Sep 06 00:56:26.875003 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:26.874976    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n"]
Sep 06 00:56:26.877275 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:26.877245    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7cfb069_044d_4179_9a64_6f3466349795.slice/crio-17c3b6ff5f0f3be510649f184b6d9c48cfb70eb409e85c0c763581ce95d6944d WatchSource:0}: Error finding container 17c3b6ff5f0f3be510649f184b6d9c48cfb70eb409e85c0c763581ce95d6944d: Status 404 returned error can't find the container with id 17c3b6ff5f0f3be510649f184b6d9c48cfb70eb409e85c0c763581ce95d6944d
Sep 06 00:56:27.771766 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:27.771729    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n" event={"ID":"a7cfb069-044d-4179-9a64-6f3466349795","Type":"ContainerStarted","Data":"17c3b6ff5f0f3be510649f184b6d9c48cfb70eb409e85c0c763581ce95d6944d"}
Sep 06 00:56:29.783046 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:29.783007    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n" event={"ID":"a7cfb069-044d-4179-9a64-6f3466349795","Type":"ContainerStarted","Data":"c7fc54c27acb49553e28649070929523ae93ea1371b8d54a754a985b3985af6e"}
Sep 06 00:56:29.799581 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:29.799528    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trx9n" podStartSLOduration=1.6815731779999998 podStartE2EDuration="3.799513978s" totalImagesPullingTime="2.1179408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:56:29.797749631 +0000 UTC m=+352.127108467" watchObservedRunningTime="2026-09-06 00:56:29.799513978 +0000 UTC m=+352.128872814"
Sep 06 00:56:30.473384 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:30.473351    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:30.574238 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:30.574204    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:31.994791 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:31.994754    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"]
Sep 06 00:56:31.998084 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:31.998067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.000613 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.000587    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 00:56:32.000613 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.000600    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 00:56:32.000803 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.000642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l54wx\""
Sep 06 00:56:32.009142 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.009118    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"]
Sep 06 00:56:32.114362 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.114331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz4sp\" (UniqueName: \"kubernetes.io/projected/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-kube-api-access-rz4sp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.114532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.114372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.114532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.114390    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.215114 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.215070    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz4sp\" (UniqueName: \"kubernetes.io/projected/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-kube-api-access-rz4sp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.215301 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.215126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.215301 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.215148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.215566 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.215543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.215606 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.215551    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.223824 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.223790    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz4sp\" (UniqueName: \"kubernetes.io/projected/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-kube-api-access-rz4sp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.306498 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.306420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:32.580818 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.580792    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"]
Sep 06 00:56:32.582688 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:32.582664    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda621ddd9_689a_4206_9bfc_1121a2bdeb8b.slice/crio-cc3a589d21a6e06760fed86273014c7725d44f636a46f35faf3226c0ff331efd WatchSource:0}: Error finding container cc3a589d21a6e06760fed86273014c7725d44f636a46f35faf3226c0ff331efd: Status 404 returned error can't find the container with id cc3a589d21a6e06760fed86273014c7725d44f636a46f35faf3226c0ff331efd
Sep 06 00:56:32.596370 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.596341    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:32.797288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.797257    2575 generic.go:356] "Generic (PLEG): container finished" podID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerID="0f0ac78ede280726f906d94b6c7677c27de2445bcba021b19c46e8f78a22b89a" exitCode=0
Sep 06 00:56:32.797456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.797306    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll" event={"ID":"a621ddd9-689a-4206-9bfc-1121a2bdeb8b","Type":"ContainerDied","Data":"0f0ac78ede280726f906d94b6c7677c27de2445bcba021b19c46e8f78a22b89a"}
Sep 06 00:56:32.797456 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:32.797340    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll" event={"ID":"a621ddd9-689a-4206-9bfc-1121a2bdeb8b","Type":"ContainerStarted","Data":"cc3a589d21a6e06760fed86273014c7725d44f636a46f35faf3226c0ff331efd"}
Sep 06 00:56:33.804333 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:33.804302    2575 generic.go:356] "Generic (PLEG): container finished" podID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerID="0afd2f128dca18ad6914f4ae3f960325ef4f2a5a60b461a51f28cd616acc6d9d" exitCode=0
Sep 06 00:56:33.804777 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:33.804375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll" event={"ID":"a621ddd9-689a-4206-9bfc-1121a2bdeb8b","Type":"ContainerDied","Data":"0afd2f128dca18ad6914f4ae3f960325ef4f2a5a60b461a51f28cd616acc6d9d"}
Sep 06 00:56:34.811260 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:34.811219    2575 generic.go:356] "Generic (PLEG): container finished" podID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerID="63e776032eab9f44617b05a69a5212b9c96802389ce007371b2cf0b427d53903" exitCode=0
Sep 06 00:56:34.811639 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:34.811285    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll" event={"ID":"a621ddd9-689a-4206-9bfc-1121a2bdeb8b","Type":"ContainerDied","Data":"63e776032eab9f44617b05a69a5212b9c96802389ce007371b2cf0b427d53903"}
Sep 06 00:56:35.934427 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:35.934403    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:36.046617 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.046573    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-util\") pod \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") "
Sep 06 00:56:36.046791 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.046636    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-bundle\") pod \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") "
Sep 06 00:56:36.046791 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.046695    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz4sp\" (UniqueName: \"kubernetes.io/projected/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-kube-api-access-rz4sp\") pod \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\" (UID: \"a621ddd9-689a-4206-9bfc-1121a2bdeb8b\") "
Sep 06 00:56:36.047297 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.047273    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-bundle" (OuterVolumeSpecName: "bundle") pod "a621ddd9-689a-4206-9bfc-1121a2bdeb8b" (UID: "a621ddd9-689a-4206-9bfc-1121a2bdeb8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:36.048908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.048883    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-kube-api-access-rz4sp" (OuterVolumeSpecName: "kube-api-access-rz4sp") pod "a621ddd9-689a-4206-9bfc-1121a2bdeb8b" (UID: "a621ddd9-689a-4206-9bfc-1121a2bdeb8b"). InnerVolumeSpecName "kube-api-access-rz4sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:56:36.052294 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.052262    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-util" (OuterVolumeSpecName: "util") pod "a621ddd9-689a-4206-9bfc-1121a2bdeb8b" (UID: "a621ddd9-689a-4206-9bfc-1121a2bdeb8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:36.148138 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.148055    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:36.148138 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.148086    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz4sp\" (UniqueName: \"kubernetes.io/projected/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-kube-api-access-rz4sp\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:36.148138 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.148099    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a621ddd9-689a-4206-9bfc-1121a2bdeb8b-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:36.822624 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.822600    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll"
Sep 06 00:56:36.822624 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.822606    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpzwll" event={"ID":"a621ddd9-689a-4206-9bfc-1121a2bdeb8b","Type":"ContainerDied","Data":"cc3a589d21a6e06760fed86273014c7725d44f636a46f35faf3226c0ff331efd"}
Sep 06 00:56:36.822825 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:36.822653    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc3a589d21a6e06760fed86273014c7725d44f636a46f35faf3226c0ff331efd"
Sep 06 00:56:42.810932 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.810892    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"]
Sep 06 00:56:42.811377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811274    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="util"
Sep 06 00:56:42.811377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811291    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="util"
Sep 06 00:56:42.811377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811302    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="extract"
Sep 06 00:56:42.811377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811310    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="extract"
Sep 06 00:56:42.811377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811340    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="pull"
Sep 06 00:56:42.811377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811348    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="pull"
Sep 06 00:56:42.811677 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.811428    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a621ddd9-689a-4206-9bfc-1121a2bdeb8b" containerName="extract"
Sep 06 00:56:42.815788 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.815767    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:42.818050 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.818029    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 00:56:42.818722 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.818708    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 00:56:42.818722 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.818715    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l54wx\""
Sep 06 00:56:42.826403 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.826384    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"]
Sep 06 00:56:42.898714 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.898689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:42.898875 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.898722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:42.898875 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.898743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp7sh\" (UniqueName: \"kubernetes.io/projected/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-kube-api-access-qp7sh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.000017 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:42.999988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.000140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.000020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.000140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.000044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp7sh\" (UniqueName: \"kubernetes.io/projected/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-kube-api-access-qp7sh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.000383 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.000363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.000441 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.000382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.007817 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.007795    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp7sh\" (UniqueName: \"kubernetes.io/projected/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-kube-api-access-qp7sh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.124342 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.124274    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:43.395709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.395682    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"]
Sep 06 00:56:43.398199 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:43.398170    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode111c91c_81b1_4a2c_8efc_f9f3c388ade3.slice/crio-0b5c12675a2e42495b505251efb24e93e69ded3920941b68814eb389d92fcdd1 WatchSource:0}: Error finding container 0b5c12675a2e42495b505251efb24e93e69ded3920941b68814eb389d92fcdd1: Status 404 returned error can't find the container with id 0b5c12675a2e42495b505251efb24e93e69ded3920941b68814eb389d92fcdd1
Sep 06 00:56:43.854324 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.854296    2575 generic.go:356] "Generic (PLEG): container finished" podID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerID="bb2244b444f88f47bdb09734aa82c2f021665d0ff8854d524f2aaa292d045abe" exitCode=0
Sep 06 00:56:43.854678 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.854330    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd" event={"ID":"e111c91c-81b1-4a2c-8efc-f9f3c388ade3","Type":"ContainerDied","Data":"bb2244b444f88f47bdb09734aa82c2f021665d0ff8854d524f2aaa292d045abe"}
Sep 06 00:56:43.854678 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:43.854364    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd" event={"ID":"e111c91c-81b1-4a2c-8efc-f9f3c388ade3","Type":"ContainerStarted","Data":"0b5c12675a2e42495b505251efb24e93e69ded3920941b68814eb389d92fcdd1"}
Sep 06 00:56:44.105658 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:44.105547    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:44.199647 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:44.199609    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:44.816732 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:44.816682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:44.860425 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:44.860399    2575 generic.go:356] "Generic (PLEG): container finished" podID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerID="6273fac8013db167fbed235f6294682cdfb3c8e4f9d4ef2303e733ebdae52009" exitCode=0
Sep 06 00:56:44.860772 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:44.860436    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd" event={"ID":"e111c91c-81b1-4a2c-8efc-f9f3c388ade3","Type":"ContainerDied","Data":"6273fac8013db167fbed235f6294682cdfb3c8e4f9d4ef2303e733ebdae52009"}
Sep 06 00:56:45.088979 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.088892    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.180579 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.180550    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.234664 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.233247    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"]
Sep 06 00:56:45.237699 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.237681    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.242098 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.242067    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 06 00:56:45.242773 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.242753    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 06 00:56:45.243117 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.243096    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fcj4z\""
Sep 06 00:56:45.243234 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.243104    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 06 00:56:45.243234 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.243203    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 06 00:56:45.260055 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.260030    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"]
Sep 06 00:56:45.319353 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.319324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/266694cc-ad09-4e8f-a7ad-bb3a0172564e-manifests\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.319493 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.319395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drqvb\" (UniqueName: \"kubernetes.io/projected/266694cc-ad09-4e8f-a7ad-bb3a0172564e-kube-api-access-drqvb\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.319493 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.319421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266694cc-ad09-4e8f-a7ad-bb3a0172564e-tmp\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.319493 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.319467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/266694cc-ad09-4e8f-a7ad-bb3a0172564e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.319646 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.319539    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266694cc-ad09-4e8f-a7ad-bb3a0172564e-webhook-cert\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.420822 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.420724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drqvb\" (UniqueName: \"kubernetes.io/projected/266694cc-ad09-4e8f-a7ad-bb3a0172564e-kube-api-access-drqvb\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.420822 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.420781    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266694cc-ad09-4e8f-a7ad-bb3a0172564e-tmp\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.421048 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.420898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/266694cc-ad09-4e8f-a7ad-bb3a0172564e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.421048 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.420979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266694cc-ad09-4e8f-a7ad-bb3a0172564e-webhook-cert\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.421048 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.421027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/266694cc-ad09-4e8f-a7ad-bb3a0172564e-manifests\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.421393 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.421365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/266694cc-ad09-4e8f-a7ad-bb3a0172564e-manifests\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.423197 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.423177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266694cc-ad09-4e8f-a7ad-bb3a0172564e-tmp\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.423753 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.423717    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/266694cc-ad09-4e8f-a7ad-bb3a0172564e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.424046 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.423749    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266694cc-ad09-4e8f-a7ad-bb3a0172564e-webhook-cert\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.424046 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.423773    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.432325 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.432304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drqvb\" (UniqueName: \"kubernetes.io/projected/266694cc-ad09-4e8f-a7ad-bb3a0172564e-kube-api-access-drqvb\") pod \"opendatahub-operator-controller-manager-79dff7ddb6-z8wrb\" (UID: \"266694cc-ad09-4e8f-a7ad-bb3a0172564e\") " pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.526126 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.526089    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.547718 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.547695    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:45.627785 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.627755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.723291 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.723255    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.802312 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.802284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:45.822383 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.822361    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"]
Sep 06 00:56:45.824945 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:56:45.824919    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266694cc_ad09_4e8f_a7ad_bb3a0172564e.slice/crio-1fb34803682e8538cc7acf1ebb33e89689794a46df705cff6679e0c4d036f72e WatchSource:0}: Error finding container 1fb34803682e8538cc7acf1ebb33e89689794a46df705cff6679e0c4d036f72e: Status 404 returned error can't find the container with id 1fb34803682e8538cc7acf1ebb33e89689794a46df705cff6679e0c4d036f72e
Sep 06 00:56:45.866900 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.866844    2575 generic.go:356] "Generic (PLEG): container finished" podID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerID="bce9eea9d9bfdb24f4c2ba0041982c8959639a34bc886196793af1cd4004c64c" exitCode=0
Sep 06 00:56:45.866900 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.866885    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd" event={"ID":"e111c91c-81b1-4a2c-8efc-f9f3c388ade3","Type":"ContainerDied","Data":"bce9eea9d9bfdb24f4c2ba0041982c8959639a34bc886196793af1cd4004c64c"}
Sep 06 00:56:45.868180 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:45.868162    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb" event={"ID":"266694cc-ad09-4e8f-a7ad-bb3a0172564e","Type":"ContainerStarted","Data":"1fb34803682e8538cc7acf1ebb33e89689794a46df705cff6679e0c4d036f72e"}
Sep 06 00:56:47.024783 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.024748    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:47.136836 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.136740    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp7sh\" (UniqueName: \"kubernetes.io/projected/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-kube-api-access-qp7sh\") pod \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") "
Sep 06 00:56:47.136836 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.136822    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-util\") pod \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") "
Sep 06 00:56:47.137083 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.136848    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-bundle\") pod \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\" (UID: \"e111c91c-81b1-4a2c-8efc-f9f3c388ade3\") "
Sep 06 00:56:47.138044 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.138008    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-bundle" (OuterVolumeSpecName: "bundle") pod "e111c91c-81b1-4a2c-8efc-f9f3c388ade3" (UID: "e111c91c-81b1-4a2c-8efc-f9f3c388ade3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:47.139671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.139642    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-kube-api-access-qp7sh" (OuterVolumeSpecName: "kube-api-access-qp7sh") pod "e111c91c-81b1-4a2c-8efc-f9f3c388ade3" (UID: "e111c91c-81b1-4a2c-8efc-f9f3c388ade3"). InnerVolumeSpecName "kube-api-access-qp7sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:56:47.145706 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.145682    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-util" (OuterVolumeSpecName: "util") pod "e111c91c-81b1-4a2c-8efc-f9f3c388ade3" (UID: "e111c91c-81b1-4a2c-8efc-f9f3c388ade3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:56:47.237904 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.237870    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp7sh\" (UniqueName: \"kubernetes.io/projected/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-kube-api-access-qp7sh\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:47.237904 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.237905    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:47.238108 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.237921    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e111c91c-81b1-4a2c-8efc-f9f3c388ade3-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:56:47.881090 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.881056    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd" event={"ID":"e111c91c-81b1-4a2c-8efc-f9f3c388ade3","Type":"ContainerDied","Data":"0b5c12675a2e42495b505251efb24e93e69ded3920941b68814eb389d92fcdd1"}
Sep 06 00:56:47.881225 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.881107    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b5c12675a2e42495b505251efb24e93e69ded3920941b68814eb389d92fcdd1"
Sep 06 00:56:47.881225 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:47.881118    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0kcwxd"
Sep 06 00:56:49.891799 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:49.891761    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb" event={"ID":"266694cc-ad09-4e8f-a7ad-bb3a0172564e","Type":"ContainerStarted","Data":"88a19c5ea4f0d41ef4a2a614a55efc35d4cf4ae30b64bcba7447b5c96245e956"}
Sep 06 00:56:50.898019 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:50.897985    2575 generic.go:356] "Generic (PLEG): container finished" podID="266694cc-ad09-4e8f-a7ad-bb3a0172564e" containerID="88a19c5ea4f0d41ef4a2a614a55efc35d4cf4ae30b64bcba7447b5c96245e956" exitCode=0
Sep 06 00:56:50.898384 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:50.898068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb" event={"ID":"266694cc-ad09-4e8f-a7ad-bb3a0172564e","Type":"ContainerDied","Data":"88a19c5ea4f0d41ef4a2a614a55efc35d4cf4ae30b64bcba7447b5c96245e956"}
Sep 06 00:56:51.905097 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:51.905062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb" event={"ID":"266694cc-ad09-4e8f-a7ad-bb3a0172564e","Type":"ContainerStarted","Data":"4ba570bd5c56775f21502982ec9f9599ab526f8332c58b6bbdf92cdd2a32628c"}
Sep 06 00:56:51.905492 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:51.905184    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:56:51.928023 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:51.927975    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb" podStartSLOduration=2.4532624800000002 podStartE2EDuration="6.927960252s" totalImagesPullingTime="3.430249232s" totalInitContainerRuntime="1.04444854s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:56:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:56:51.925654541 +0000 UTC m=+374.255013396" watchObservedRunningTime="2026-09-06 00:56:51.927960252 +0000 UTC m=+374.257319087"
Sep 06 00:56:54.706495 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:54.706460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:58.911458 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:58.911418    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:56:59.012124 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:56:59.012085    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:01.839798 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:01.839764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:01.929672 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:01.929638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:02.261867 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:02.261827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:02.319935 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:02.319902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:02.912440 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:02.912405    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79dff7ddb6-z8wrb"
Sep 06 00:57:04.594165 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:04.594121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:04.781988 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:04.781946    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:10.236750 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.236713    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:10.254234 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254205    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"]
Sep 06 00:57:10.254491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254479    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="pull"
Sep 06 00:57:10.254532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254494    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="pull"
Sep 06 00:57:10.254532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254508    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="extract"
Sep 06 00:57:10.254532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254513    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="extract"
Sep 06 00:57:10.254532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254524    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="util"
Sep 06 00:57:10.254532 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254529    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="util"
Sep 06 00:57:10.254672 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.254582    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e111c91c-81b1-4a2c-8efc-f9f3c388ade3" containerName="extract"
Sep 06 00:57:10.258840 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.258822    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.262255 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.262235    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 06 00:57:10.262367 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.262273    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 06 00:57:10.262367 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.262297    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 06 00:57:10.262367 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.262356    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hvxhq\""
Sep 06 00:57:10.262845 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.262829    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 06 00:57:10.268879 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.268841    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"]
Sep 06 00:57:10.306848 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.306821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9946e0fe-8689-4a4e-866a-9f5007834772-tmp\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.306962 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.306877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9946e0fe-8689-4a4e-866a-9f5007834772-tls-certs\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.306962 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.306944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk7wp\" (UniqueName: \"kubernetes.io/projected/9946e0fe-8689-4a4e-866a-9f5007834772-kube-api-access-hk7wp\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.407529 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.407505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9946e0fe-8689-4a4e-866a-9f5007834772-tmp\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.407634 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.407538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9946e0fe-8689-4a4e-866a-9f5007834772-tls-certs\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.407634 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.407562    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk7wp\" (UniqueName: \"kubernetes.io/projected/9946e0fe-8689-4a4e-866a-9f5007834772-kube-api-access-hk7wp\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.407712 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:10.407643    2575 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 06 00:57:10.407712 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:10.407699    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9946e0fe-8689-4a4e-866a-9f5007834772-tls-certs podName:9946e0fe-8689-4a4e-866a-9f5007834772 nodeName:}" failed. No retries permitted until 2026-09-06 00:57:10.907683395 +0000 UTC m=+393.237042209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9946e0fe-8689-4a4e-866a-9f5007834772-tls-certs") pod "kube-auth-proxy-5cb67ffb98-z4cth" (UID: "9946e0fe-8689-4a4e-866a-9f5007834772") : secret "kube-auth-proxy-tls" not found
Sep 06 00:57:10.409757 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.409729    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9946e0fe-8689-4a4e-866a-9f5007834772-tmp\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.417836 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.417806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk7wp\" (UniqueName: \"kubernetes.io/projected/9946e0fe-8689-4a4e-866a-9f5007834772-kube-api-access-hk7wp\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.538358 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.538283    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:10.912148 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.912069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9946e0fe-8689-4a4e-866a-9f5007834772-tls-certs\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:10.914568 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:10.914532    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9946e0fe-8689-4a4e-866a-9f5007834772-tls-certs\") pod \"kube-auth-proxy-5cb67ffb98-z4cth\" (UID: \"9946e0fe-8689-4a4e-866a-9f5007834772\") " pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:11.168582 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:11.168501    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"
Sep 06 00:57:11.452215 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:11.452192    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth"]
Sep 06 00:57:11.454587 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:57:11.454564    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9946e0fe_8689_4a4e_866a_9f5007834772.slice/crio-24d10ab959d3f392d360f5d163b8f00614f0b5db9b640b39f189e52935a2a1ee WatchSource:0}: Error finding container 24d10ab959d3f392d360f5d163b8f00614f0b5db9b640b39f189e52935a2a1ee: Status 404 returned error can't find the container with id 24d10ab959d3f392d360f5d163b8f00614f0b5db9b640b39f189e52935a2a1ee
Sep 06 00:57:11.530241 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:11.530213    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:11.596539 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:11.596501    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:11.992868 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:11.992823    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth" event={"ID":"9946e0fe-8689-4a4e-866a-9f5007834772","Type":"ContainerStarted","Data":"24d10ab959d3f392d360f5d163b8f00614f0b5db9b640b39f189e52935a2a1ee"}
Sep 06 00:57:12.478788 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:12.478745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:15.007619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.007586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth" event={"ID":"9946e0fe-8689-4a4e-866a-9f5007834772","Type":"ContainerStarted","Data":"f7e6f2837f035bb984fc51c72407a1bbb18387aa39874e85e3f44f4ef0da8c68"}
Sep 06 00:57:15.025757 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.025711    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cb67ffb98-z4cth" podStartSLOduration=2.356458547 podStartE2EDuration="5.025694201s" totalImagesPullingTime="2.669235654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:57:15.023500982 +0000 UTC m=+397.352859818" watchObservedRunningTime="2026-09-06 00:57:15.025694201 +0000 UTC m=+397.355053037"
Sep 06 00:57:15.692277 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.692248    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2qs97"]
Sep 06 00:57:15.695294 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.695279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:15.699927 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.699901    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 06 00:57:15.700069 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.700055    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vn57b\""
Sep 06 00:57:15.707426 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.707407    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2qs97"]
Sep 06 00:57:15.725463 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.725435    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:15.754671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.754638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4155d25e-a9fd-40e0-b077-7558ef789d00-cert\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:15.754795 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.754699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cztpd\" (UniqueName: \"kubernetes.io/projected/4155d25e-a9fd-40e0-b077-7558ef789d00-kube-api-access-cztpd\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:15.823931 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.823901    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:15.856041 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.856012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4155d25e-a9fd-40e0-b077-7558ef789d00-cert\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:15.856170 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.856055    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cztpd\" (UniqueName: \"kubernetes.io/projected/4155d25e-a9fd-40e0-b077-7558ef789d00-kube-api-access-cztpd\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:15.856210 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:15.856164    2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 06 00:57:15.856253 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:15.856242    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4155d25e-a9fd-40e0-b077-7558ef789d00-cert podName:4155d25e-a9fd-40e0-b077-7558ef789d00 nodeName:}" failed. No retries permitted until 2026-09-06 00:57:16.356224891 +0000 UTC m=+398.685583716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4155d25e-a9fd-40e0-b077-7558ef789d00-cert") pod "odh-model-controller-7998c67dc4-2qs97" (UID: "4155d25e-a9fd-40e0-b077-7558ef789d00") : secret "odh-model-controller-webhook-cert" not found
Sep 06 00:57:15.867355 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.867336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cztpd\" (UniqueName: \"kubernetes.io/projected/4155d25e-a9fd-40e0-b077-7558ef789d00-kube-api-access-cztpd\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:15.922386 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:15.922350    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:16.022615 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.022591    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:16.089386 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.089347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:16.153570 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.153543    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:16.271214 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.271184    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f8zs7"]
Sep 06 00:57:16.274400 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.274360    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.276671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.276650    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 06 00:57:16.276831 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.276723    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:16.287608 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.287582    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f8zs7"]
Sep 06 00:57:16.360125 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.360092    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwk6n\" (UniqueName: \"kubernetes.io/projected/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-kube-api-access-lwk6n\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.360289 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.360148    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-tls-certs\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.360289 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.360236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4155d25e-a9fd-40e0-b077-7558ef789d00-cert\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:16.362638 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.362621    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4155d25e-a9fd-40e0-b077-7558ef789d00-cert\") pod \"odh-model-controller-7998c67dc4-2qs97\" (UID: \"4155d25e-a9fd-40e0-b077-7558ef789d00\") " pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:16.461137 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.461104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwk6n\" (UniqueName: \"kubernetes.io/projected/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-kube-api-access-lwk6n\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.461281 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.461171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-tls-certs\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.461326 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:16.461289    2575 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 06 00:57:16.461360 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:16.461347    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-tls-certs podName:10b93760-2d2d-4c96-a1a4-df2830bf5ab9 nodeName:}" failed. No retries permitted until 2026-09-06 00:57:16.961327571 +0000 UTC m=+399.290686386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-tls-certs") pod "model-serving-api-54d89bd5fc-f8zs7" (UID: "10b93760-2d2d-4c96-a1a4-df2830bf5ab9") : secret "model-serving-api-tls" not found
Sep 06 00:57:16.472397 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.472374    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwk6n\" (UniqueName: \"kubernetes.io/projected/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-kube-api-access-lwk6n\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.605150 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.605086    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:16.672059 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.672027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:16.878473 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.878451    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2qs97"]
Sep 06 00:57:16.880542 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:57:16.880510    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4155d25e_a9fd_40e0_b077_7558ef789d00.slice/crio-e938d73ab5f9f2e711df435e6d65e4973829722c695f4470bcce9d3f8c877611 WatchSource:0}: Error finding container e938d73ab5f9f2e711df435e6d65e4973829722c695f4470bcce9d3f8c877611: Status 404 returned error can't find the container with id e938d73ab5f9f2e711df435e6d65e4973829722c695f4470bcce9d3f8c877611
Sep 06 00:57:16.955958 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.955930    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"]
Sep 06 00:57:16.960504 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.960489    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:16.962888 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.962866    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-njn84\""
Sep 06 00:57:16.962966 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.962866    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 00:57:16.964747 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.964724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-tls-certs\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.967465 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.967235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b93760-2d2d-4c96-a1a4-df2830bf5ab9-tls-certs\") pod \"model-serving-api-54d89bd5fc-f8zs7\" (UID: \"10b93760-2d2d-4c96-a1a4-df2830bf5ab9\") " pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:16.971584 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.971565    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"]
Sep 06 00:57:16.988754 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:16.988722    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:17.017567 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.017540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97" event={"ID":"4155d25e-a9fd-40e0-b077-7558ef789d00","Type":"ContainerStarted","Data":"e938d73ab5f9f2e711df435e6d65e4973829722c695f4470bcce9d3f8c877611"}
Sep 06 00:57:17.065933 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.065899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rvn8\" (UniqueName: \"kubernetes.io/projected/a3175541-f613-4d89-af83-da8456e3bff1-kube-api-access-8rvn8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.065939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.065973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3175541-f613-4d89-af83-da8456e3bff1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.066008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.066033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.066047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3175541-f613-4d89-af83-da8456e3bff1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.066090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3175541-f613-4d89-af83-da8456e3bff1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.066149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.066288 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.066166    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167464 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rvn8\" (UniqueName: \"kubernetes.io/projected/a3175541-f613-4d89-af83-da8456e3bff1-kube-api-access-8rvn8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167464 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167464 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3175541-f613-4d89-af83-da8456e3bff1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167673 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167710 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167657    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167710 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3175541-f613-4d89-af83-da8456e3bff1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167785 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167741    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3175541-f613-4d89-af83-da8456e3bff1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167834 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167792    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167834 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.167958 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.168012 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.167987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.168206 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.168183    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.168283 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.168245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.168464 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.168445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3175541-f613-4d89-af83-da8456e3bff1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.169786 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.169764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3175541-f613-4d89-af83-da8456e3bff1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.170336 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.170306    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3175541-f613-4d89-af83-da8456e3bff1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.180827 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.180802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rvn8\" (UniqueName: \"kubernetes.io/projected/a3175541-f613-4d89-af83-da8456e3bff1-kube-api-access-8rvn8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.180924 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.180823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3175541-f613-4d89-af83-da8456e3bff1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc\" (UID: \"a3175541-f613-4d89-af83-da8456e3bff1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.184671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.184656    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:17.260018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.259905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:17.277416 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.277389    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:17.455843 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.455816    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f8zs7"]
Sep 06 00:57:17.457438 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:57:17.457398    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b93760_2d2d_4c96_a1a4_df2830bf5ab9.slice/crio-043bf976446a2b3071e7720caa1999d395bfbd3221b388da1785e3189eb7a989 WatchSource:0}: Error finding container 043bf976446a2b3071e7720caa1999d395bfbd3221b388da1785e3189eb7a989: Status 404 returned error can't find the container with id 043bf976446a2b3071e7720caa1999d395bfbd3221b388da1785e3189eb7a989
Sep 06 00:57:17.458303 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.458276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:17.554671 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.554649    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"]
Sep 06 00:57:17.556664 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:57:17.556638    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3175541_f613_4d89_af83_da8456e3bff1.slice/crio-13d8f12a391e4a7695adaac20030457efa0ef06dc79d57805f74398ebfc63352 WatchSource:0}: Error finding container 13d8f12a391e4a7695adaac20030457efa0ef06dc79d57805f74398ebfc63352: Status 404 returned error can't find the container with id 13d8f12a391e4a7695adaac20030457efa0ef06dc79d57805f74398ebfc63352
Sep 06 00:57:17.652576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.652550    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9765c7c-xm4km"]
Sep 06 00:57:17.657172 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.657154    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.667069 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.667046    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9765c7c-xm4km"]
Sep 06 00:57:17.671165 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59nm9\" (UniqueName: \"kubernetes.io/projected/297eefe8-3829-4c27-b438-c2ff193162c9-kube-api-access-59nm9\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.671280 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/297eefe8-3829-4c27-b438-c2ff193162c9-console-serving-cert\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.671280 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-service-ca\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.671280 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671270    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-oauth-serving-cert\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.671445 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/297eefe8-3829-4c27-b438-c2ff193162c9-console-oauth-config\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.671445 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-console-config\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.671445 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.671372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-trusted-ca-bundle\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.708311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.708235    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:17.772190 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-oauth-serving-cert\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.772321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/297eefe8-3829-4c27-b438-c2ff193162c9-console-oauth-config\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.772321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-console-config\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.772321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-trusted-ca-bundle\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.772321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772294    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59nm9\" (UniqueName: \"kubernetes.io/projected/297eefe8-3829-4c27-b438-c2ff193162c9-kube-api-access-59nm9\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.772321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/297eefe8-3829-4c27-b438-c2ff193162c9-console-serving-cert\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.772528 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.772339    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-service-ca\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.773062 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.773032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-oauth-serving-cert\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.773168 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.773114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-trusted-ca-bundle\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.773168 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.773122    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-console-config\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.773168 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.773133    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/297eefe8-3829-4c27-b438-c2ff193162c9-service-ca\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.774760 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.774740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/297eefe8-3829-4c27-b438-c2ff193162c9-console-oauth-config\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.774986 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.774971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/297eefe8-3829-4c27-b438-c2ff193162c9-console-serving-cert\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.779941 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.779917    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59nm9\" (UniqueName: \"kubernetes.io/projected/297eefe8-3829-4c27-b438-c2ff193162c9-kube-api-access-59nm9\") pod \"console-c9765c7c-xm4km\" (UID: \"297eefe8-3829-4c27-b438-c2ff193162c9\") " pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:17.967715 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:17.967647    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:18.032220 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:18.032131    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7" event={"ID":"10b93760-2d2d-4c96-a1a4-df2830bf5ab9","Type":"ContainerStarted","Data":"043bf976446a2b3071e7720caa1999d395bfbd3221b388da1785e3189eb7a989"}
Sep 06 00:57:18.037404 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:18.037330    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc" event={"ID":"a3175541-f613-4d89-af83-da8456e3bff1","Type":"ContainerStarted","Data":"13d8f12a391e4a7695adaac20030457efa0ef06dc79d57805f74398ebfc63352"}
Sep 06 00:57:18.263237 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:18.263206    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9765c7c-xm4km"]
Sep 06 00:57:18.271195 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:57:18.271168    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297eefe8_3829_4c27_b438_c2ff193162c9.slice/crio-35b13bf21735adb50213a3861a372a8a79949b865f569bad85c95f4e633450c7 WatchSource:0}: Error finding container 35b13bf21735adb50213a3861a372a8a79949b865f569bad85c95f4e633450c7: Status 404 returned error can't find the container with id 35b13bf21735adb50213a3861a372a8a79949b865f569bad85c95f4e633450c7
Sep 06 00:57:19.045257 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:19.045217    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9765c7c-xm4km" event={"ID":"297eefe8-3829-4c27-b438-c2ff193162c9","Type":"ContainerStarted","Data":"1c0972d8ad74ddb32bdef6fed308eaa74211cc6e7cc45015ea1c4507a781d8ad"}
Sep 06 00:57:19.045457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:19.045280    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9765c7c-xm4km" event={"ID":"297eefe8-3829-4c27-b438-c2ff193162c9","Type":"ContainerStarted","Data":"35b13bf21735adb50213a3861a372a8a79949b865f569bad85c95f4e633450c7"}
Sep 06 00:57:19.548742 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:19.548700    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:19.606553 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:19.606507    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:20.646615 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:20.646575    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:20.703581 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:20.703553    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:21.632268 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:21.632224    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 00:57:21.632359 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:21.632301    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 00:57:21.632359 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:21.632332    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 00:57:22.065000 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.064896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97" event={"ID":"4155d25e-a9fd-40e0-b077-7558ef789d00","Type":"ContainerStarted","Data":"887b5c465b63f4f4c9194371196423af8bfd2ace7b9fae70824e8da3c0176fa0"}
Sep 06 00:57:22.065457 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.065098    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:22.066757 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.066733    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7" event={"ID":"10b93760-2d2d-4c96-a1a4-df2830bf5ab9","Type":"ContainerStarted","Data":"8c78eb56de81d85548680f1f8036524cb0bab7d0c2e8bb751f5de574246ecaec"}
Sep 06 00:57:22.066914 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.066891    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:22.068649 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.068626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc" event={"ID":"a3175541-f613-4d89-af83-da8456e3bff1","Type":"ContainerStarted","Data":"5c84f1ef672d127952bf632f11e371e02d484e0937f8ec736e7306b68d50f2d6"}
Sep 06 00:57:22.085268 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.085224    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97" podStartSLOduration=2.342076343 podStartE2EDuration="7.085207853s" totalImagesPullingTime="4.74313151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:57:22.084169696 +0000 UTC m=+404.413528543" watchObservedRunningTime="2026-09-06 00:57:22.085207853 +0000 UTC m=+404.414566692"
Sep 06 00:57:22.085846 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.085815    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9765c7c-xm4km" podStartSLOduration=5.08580295 podStartE2EDuration="5.08580295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:57:19.064046137 +0000 UTC m=+401.393404974" watchObservedRunningTime="2026-09-06 00:57:22.08580295 +0000 UTC m=+404.415161783"
Sep 06 00:57:22.104285 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.104240    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc" podStartSLOduration=2.030688618 podStartE2EDuration="6.104225572s" totalImagesPullingTime="4.073536954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:57:22.103530445 +0000 UTC m=+404.432889303" watchObservedRunningTime="2026-09-06 00:57:22.104225572 +0000 UTC m=+404.433584408"
Sep 06 00:57:22.122295 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.122252    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7" podStartSLOduration=1.955458793 podStartE2EDuration="6.12223685s" totalImagesPullingTime="4.166778057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:57:22.120569036 +0000 UTC m=+404.449927881" watchObservedRunningTime="2026-09-06 00:57:22.12223685 +0000 UTC m=+404.451595685"
Sep 06 00:57:22.278333 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.278301    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:22.283059 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:22.283036    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:23.074651 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:23.074621    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:23.075606 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:23.075583    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc"
Sep 06 00:57:25.253577 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.253544    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"]
Sep 06 00:57:25.259007 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.258983    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.262513 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.262495    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 06 00:57:25.263284 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.263267    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 06 00:57:25.263381 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.263293    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hxwss\""
Sep 06 00:57:25.264232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.264214    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 06 00:57:25.286795 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.286777    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"]
Sep 06 00:57:25.340526 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.340499    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a240ca61-e040-41d5-873d-ffbe0d8b682d-metrics-cert\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.340649 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.340549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqrqj\" (UniqueName: \"kubernetes.io/projected/a240ca61-e040-41d5-873d-ffbe0d8b682d-kube-api-access-vqrqj\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.340649 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.340595    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a240ca61-e040-41d5-873d-ffbe0d8b682d-manager-config\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.340649 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.340609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a240ca61-e040-41d5-873d-ffbe0d8b682d-cert\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.442060 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.442025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqrqj\" (UniqueName: \"kubernetes.io/projected/a240ca61-e040-41d5-873d-ffbe0d8b682d-kube-api-access-vqrqj\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.442216 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.442077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a240ca61-e040-41d5-873d-ffbe0d8b682d-manager-config\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.442216 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.442175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a240ca61-e040-41d5-873d-ffbe0d8b682d-cert\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.442322 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.442289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a240ca61-e040-41d5-873d-ffbe0d8b682d-metrics-cert\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.442742 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.442720    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a240ca61-e040-41d5-873d-ffbe0d8b682d-manager-config\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.444763 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.444744    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a240ca61-e040-41d5-873d-ffbe0d8b682d-metrics-cert\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.444916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.444776    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a240ca61-e040-41d5-873d-ffbe0d8b682d-cert\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.457551 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.457528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqrqj\" (UniqueName: \"kubernetes.io/projected/a240ca61-e040-41d5-873d-ffbe0d8b682d-kube-api-access-vqrqj\") pod \"lws-controller-manager-c79fd5646-82vf2\" (UID: \"a240ca61-e040-41d5-873d-ffbe0d8b682d\") " pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.568139 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.568062    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:25.643704 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.643612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:25.744569 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.744508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:25.851198 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.851084    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"]
Sep 06 00:57:25.851995 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:25.851966    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:25.853486 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:57:25.853463    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda240ca61_e040_41d5_873d_ffbe0d8b682d.slice/crio-25713609f73a2c69333aa38e8feb02b7d3236de60141e63c622f5e98f3b20bbf WatchSource:0}: Error finding container 25713609f73a2c69333aa38e8feb02b7d3236de60141e63c622f5e98f3b20bbf: Status 404 returned error can't find the container with id 25713609f73a2c69333aa38e8feb02b7d3236de60141e63c622f5e98f3b20bbf
Sep 06 00:57:26.008708 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:26.008679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:26.089983 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:26.089946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2" event={"ID":"a240ca61-e040-41d5-873d-ffbe0d8b682d","Type":"ContainerStarted","Data":"25713609f73a2c69333aa38e8feb02b7d3236de60141e63c622f5e98f3b20bbf"}
Sep 06 00:57:26.950958 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:26.950879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:27.059135 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:27.059098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:27.176262 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:27.176223    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:27.968200 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:27.968163    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:27.968572 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:27.968215    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:27.972997 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:27.972977    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:28.103441 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.103403    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2" event={"ID":"a240ca61-e040-41d5-873d-ffbe0d8b682d","Type":"ContainerStarted","Data":"e48cafffda4e7f3bac3c3310a24d2acf9900c07e8c2765e4048cb86f470421a3"}
Sep 06 00:57:28.103619 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.103491    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:28.107304 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.107284    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9765c7c-xm4km"
Sep 06 00:57:28.132496 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.132441    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2" podStartSLOduration=1.6143431000000001 podStartE2EDuration="3.132427412s" totalImagesPullingTime="1.518084312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:57:28.131490412 +0000 UTC m=+410.460849248" watchObservedRunningTime="2026-09-06 00:57:28.132427412 +0000 UTC m=+410.461786247"
Sep 06 00:57:28.251721 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.248823    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744dcf646b-xstsg"]
Sep 06 00:57:28.367204 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.367170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:28.467194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.467161    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:28.567339 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:28.567254    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:32.947984 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:32.947945    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:33.002814 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:33.002776    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:33.076847 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:33.076821    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2qs97"
Sep 06 00:57:33.078824 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:33.078804    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-f8zs7"
Sep 06 00:57:36.224232 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:36.224197    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:36.325994 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:36.325951    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:57:39.111694 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:39.111659    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c79fd5646-82vf2"
Sep 06 00:57:53.279447 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.279385    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-744dcf646b-xstsg" podUID="be1f5342-72d6-4ed7-89ac-db40f3035977" containerName="console" containerID="cri-o://a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283" gracePeriod=15
Sep 06 00:57:53.514146 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.514119    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:57:53.677923 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.677821    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rzfv\" (UniqueName: \"kubernetes.io/projected/be1f5342-72d6-4ed7-89ac-db40f3035977-kube-api-access-8rzfv\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.677923 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.677884    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-service-ca\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.678127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.677930    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-serving-cert\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.678127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678008    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-console-config\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.678127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678033    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-oauth-config\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.678127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678076    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-oauth-serving-cert\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.678127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678092    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-trusted-ca-bundle\") pod \"be1f5342-72d6-4ed7-89ac-db40f3035977\" (UID: \"be1f5342-72d6-4ed7-89ac-db40f3035977\") "
Sep 06 00:57:53.678387 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678330    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-console-config" (OuterVolumeSpecName: "console-config") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:57:53.678448 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678304    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-service-ca" (OuterVolumeSpecName: "service-ca") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:57:53.678516 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678486    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:57:53.678624 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.678587    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 00:57:53.680216 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.680188    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be1f5342-72d6-4ed7-89ac-db40f3035977-kube-api-access-8rzfv" (OuterVolumeSpecName: "kube-api-access-8rzfv") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "kube-api-access-8rzfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:57:53.680322 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.680267    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:57:53.680380 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.680362    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "be1f5342-72d6-4ed7-89ac-db40f3035977" (UID: "be1f5342-72d6-4ed7-89ac-db40f3035977"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 00:57:53.779043 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779021    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:53.779043 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779041    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:53.779194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779050    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-console-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:53.779194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779059    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be1f5342-72d6-4ed7-89ac-db40f3035977-console-oauth-config\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:53.779194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779067    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-oauth-serving-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:53.779194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779075    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be1f5342-72d6-4ed7-89ac-db40f3035977-trusted-ca-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:53.779194 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:53.779084    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rzfv\" (UniqueName: \"kubernetes.io/projected/be1f5342-72d6-4ed7-89ac-db40f3035977-kube-api-access-8rzfv\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:57:54.231162 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.231127    2575 generic.go:356] "Generic (PLEG): container finished" podID="be1f5342-72d6-4ed7-89ac-db40f3035977" containerID="a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283" exitCode=0
Sep 06 00:57:54.231311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.231172    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744dcf646b-xstsg"
Sep 06 00:57:54.231311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.231215    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744dcf646b-xstsg" event={"ID":"be1f5342-72d6-4ed7-89ac-db40f3035977","Type":"ContainerDied","Data":"a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283"}
Sep 06 00:57:54.231311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.231269    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744dcf646b-xstsg" event={"ID":"be1f5342-72d6-4ed7-89ac-db40f3035977","Type":"ContainerDied","Data":"8bf01357e08411d27d0a40955bffbcef232eeb48fbf5b8810f403bc14a98174f"}
Sep 06 00:57:54.231311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.231294    2575 scope.go:117] "RemoveContainer" containerID="a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283"
Sep 06 00:57:54.239743 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.239726    2575 scope.go:117] "RemoveContainer" containerID="a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283"
Sep 06 00:57:54.240011 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:57:54.239994    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283\": container with ID starting with a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283 not found: ID does not exist" containerID="a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283"
Sep 06 00:57:54.240068 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.240018    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283"} err="failed to get container status \"a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283\": rpc error: code = NotFound desc = could not find container \"a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283\": container with ID starting with a22fa0210c17a9546a719b77c842ac613e87496ccce514e8d20332b87bb55283 not found: ID does not exist"
Sep 06 00:57:54.247535 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.247510    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744dcf646b-xstsg"]
Sep 06 00:57:54.252886 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:54.252829    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744dcf646b-xstsg"]
Sep 06 00:57:56.201175 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:57:56.201134    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be1f5342-72d6-4ed7-89ac-db40f3035977" path="/var/lib/kubelet/pods/be1f5342-72d6-4ed7-89ac-db40f3035977/volumes"
Sep 06 00:58:16.711319 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:16.711283    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:16.804788 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:16.804753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:20.334028 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.333986    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"]
Sep 06 00:58:20.334488 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.334441    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be1f5342-72d6-4ed7-89ac-db40f3035977" containerName="console"
Sep 06 00:58:20.334488 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.334459    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be1f5342-72d6-4ed7-89ac-db40f3035977" containerName="console"
Sep 06 00:58:20.334601 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.334546    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="be1f5342-72d6-4ed7-89ac-db40f3035977" containerName="console"
Sep 06 00:58:20.337761 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.337740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.340147 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.340127    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l54wx\""
Sep 06 00:58:20.340230 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.340211    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 06 00:58:20.340750 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.340735    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 06 00:58:20.346686 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.346656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"]
Sep 06 00:58:20.391606 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.391574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.391759 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.391623    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npj4j\" (UniqueName: \"kubernetes.io/projected/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-kube-api-access-npj4j\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.391759 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.391647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.431510 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.431480    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"]
Sep 06 00:58:20.434785 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.434769    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.443630 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.443607    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"]
Sep 06 00:58:20.492617 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492596    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.492754 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.492754 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5ctn\" (UniqueName: \"kubernetes.io/projected/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-kube-api-access-q5ctn\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.492754 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492709    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npj4j\" (UniqueName: \"kubernetes.io/projected/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-kube-api-access-npj4j\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.492976 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.492976 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492814    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.492976 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.492952    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.493080 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.493045    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.501269 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.501251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npj4j\" (UniqueName: \"kubernetes.io/projected/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-kube-api-access-npj4j\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.531364 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.531338    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"]
Sep 06 00:58:20.534745 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.534730    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.543279 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.543245    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"]
Sep 06 00:58:20.594209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594140    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.594209 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594189    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hctl\" (UniqueName: \"kubernetes.io/projected/90dca77a-dd41-45f4-9f77-308e66d999ba-kube-api-access-5hctl\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.594377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.594377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594265    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.594377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.594377 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594351    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5ctn\" (UniqueName: \"kubernetes.io/projected/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-kube-api-access-q5ctn\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.594531 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594519    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.594566 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.594550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.602316 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.602291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5ctn\" (UniqueName: \"kubernetes.io/projected/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-kube-api-access-q5ctn\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.632898 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.632873    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"]
Sep 06 00:58:20.636286 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.636271    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.644127 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.644105    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"]
Sep 06 00:58:20.647838 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.647774    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:20.695097 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.695256 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695116    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hctl\" (UniqueName: \"kubernetes.io/projected/90dca77a-dd41-45f4-9f77-308e66d999ba-kube-api-access-5hctl\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.695256 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.695256 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695189    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.695256 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.695437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695319    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpgcx\" (UniqueName: \"kubernetes.io/projected/ae1cfc43-1182-42c6-9952-a6757fee0d1a-kube-api-access-dpgcx\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.695560 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695541    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.695661 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.695639    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.704206 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.704157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hctl\" (UniqueName: \"kubernetes.io/projected/90dca77a-dd41-45f4-9f77-308e66d999ba-kube-api-access-5hctl\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.744487 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.744456    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:20.797438 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.797395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.797559 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.797530    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpgcx\" (UniqueName: \"kubernetes.io/projected/ae1cfc43-1182-42c6-9952-a6757fee0d1a-kube-api-access-dpgcx\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.797622 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.797582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.797782 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.797761    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.797983 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.797953    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.807185 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.807164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpgcx\" (UniqueName: \"kubernetes.io/projected/ae1cfc43-1182-42c6-9952-a6757fee0d1a-kube-api-access-dpgcx\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:20.845016 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.844955    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:20.925908 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.925883    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"]
Sep 06 00:58:20.928878 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:58:20.928837    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a9be9c_f71d_4b19_a072_db00ddd8a24c.slice/crio-165149d3ad454f2e786319104f68f18ebeeb196e9ecdd0a3dca94c54d22b663d WatchSource:0}: Error finding container 165149d3ad454f2e786319104f68f18ebeeb196e9ecdd0a3dca94c54d22b663d: Status 404 returned error can't find the container with id 165149d3ad454f2e786319104f68f18ebeeb196e9ecdd0a3dca94c54d22b663d
Sep 06 00:58:20.946745 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:20.946724    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:21.015885 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.015838    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"]
Sep 06 00:58:21.018931 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:58:21.018905    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef1a2cb_6898_48e7_95f0_a1f5caf2b0c3.slice/crio-a0061999b3b168d743265dd9bc642fe6bc32155030e54c8a2d58924fee6f2698 WatchSource:0}: Error finding container a0061999b3b168d743265dd9bc642fe6bc32155030e54c8a2d58924fee6f2698: Status 404 returned error can't find the container with id a0061999b3b168d743265dd9bc642fe6bc32155030e54c8a2d58924fee6f2698
Sep 06 00:58:21.119579 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.119555    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"]
Sep 06 00:58:21.121497 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:58:21.121471    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90dca77a_dd41_45f4_9f77_308e66d999ba.slice/crio-3d82d52bf9250657c1c3567f5a72d2edff7c5013345da9f8d890744409b9bb38 WatchSource:0}: Error finding container 3d82d52bf9250657c1c3567f5a72d2edff7c5013345da9f8d890744409b9bb38: Status 404 returned error can't find the container with id 3d82d52bf9250657c1c3567f5a72d2edff7c5013345da9f8d890744409b9bb38
Sep 06 00:58:21.223494 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.223466    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"]
Sep 06 00:58:21.256648 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:58:21.256622    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1cfc43_1182_42c6_9952_a6757fee0d1a.slice/crio-4581807b8a88967545010b3de869538500a5a32735331ebb84506746a2216958 WatchSource:0}: Error finding container 4581807b8a88967545010b3de869538500a5a32735331ebb84506746a2216958: Status 404 returned error can't find the container with id 4581807b8a88967545010b3de869538500a5a32735331ebb84506746a2216958
Sep 06 00:58:21.358615 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.358589    2575 generic.go:356] "Generic (PLEG): container finished" podID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerID="32e6c7c3172f83888a351a01ceff0487505e7407b4919725a25fedaf71b1d5da" exitCode=0
Sep 06 00:58:21.358972 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.358666    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q" event={"ID":"90dca77a-dd41-45f4-9f77-308e66d999ba","Type":"ContainerDied","Data":"32e6c7c3172f83888a351a01ceff0487505e7407b4919725a25fedaf71b1d5da"}
Sep 06 00:58:21.358972 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.358721    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q" event={"ID":"90dca77a-dd41-45f4-9f77-308e66d999ba","Type":"ContainerStarted","Data":"3d82d52bf9250657c1c3567f5a72d2edff7c5013345da9f8d890744409b9bb38"}
Sep 06 00:58:21.360465 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.360448    2575 generic.go:356] "Generic (PLEG): container finished" podID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerID="bae82a3c59020b92b261a2040186982ef26a1057716f240fdcffc15803f9d659" exitCode=0
Sep 06 00:58:21.360543 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.360474    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr" event={"ID":"e8a9be9c-f71d-4b19-a072-db00ddd8a24c","Type":"ContainerDied","Data":"bae82a3c59020b92b261a2040186982ef26a1057716f240fdcffc15803f9d659"}
Sep 06 00:58:21.360543 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.360514    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr" event={"ID":"e8a9be9c-f71d-4b19-a072-db00ddd8a24c","Type":"ContainerStarted","Data":"165149d3ad454f2e786319104f68f18ebeeb196e9ecdd0a3dca94c54d22b663d"}
Sep 06 00:58:21.362437 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.362420    2575 generic.go:356] "Generic (PLEG): container finished" podID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerID="4dc17ffdceea9aec2d0cb02a69202096b096508117d0bd7d5d7d2364a8f09a5f" exitCode=0
Sep 06 00:58:21.362582 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.362537    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg" event={"ID":"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3","Type":"ContainerDied","Data":"4dc17ffdceea9aec2d0cb02a69202096b096508117d0bd7d5d7d2364a8f09a5f"}
Sep 06 00:58:21.362695 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.362588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg" event={"ID":"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3","Type":"ContainerStarted","Data":"a0061999b3b168d743265dd9bc642fe6bc32155030e54c8a2d58924fee6f2698"}
Sep 06 00:58:21.364387 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.364365    2575 generic.go:356] "Generic (PLEG): container finished" podID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerID="4dad63e9f60e805fd6a7df126b6b3d41c05f7079f80ca51d60d32ac6999dff07" exitCode=0
Sep 06 00:58:21.364455 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.364418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v" event={"ID":"ae1cfc43-1182-42c6-9952-a6757fee0d1a","Type":"ContainerDied","Data":"4dad63e9f60e805fd6a7df126b6b3d41c05f7079f80ca51d60d32ac6999dff07"}
Sep 06 00:58:21.364455 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.364451    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v" event={"ID":"ae1cfc43-1182-42c6-9952-a6757fee0d1a","Type":"ContainerStarted","Data":"4581807b8a88967545010b3de869538500a5a32735331ebb84506746a2216958"}
Sep 06 00:58:21.871409 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.871370    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:21.928018 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:21.927984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:22.372348 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:22.372291    2575 generic.go:356] "Generic (PLEG): container finished" podID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerID="659eed6465ee71fb4b4dd236b37cbd1257a12055b4e673f9607a80d9b506bafc" exitCode=0
Sep 06 00:58:22.372650 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:22.372413    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q" event={"ID":"90dca77a-dd41-45f4-9f77-308e66d999ba","Type":"ContainerDied","Data":"659eed6465ee71fb4b4dd236b37cbd1257a12055b4e673f9607a80d9b506bafc"}
Sep 06 00:58:23.378657 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.378631    2575 generic.go:356] "Generic (PLEG): container finished" podID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerID="d7fc23a3e065f80efab672dc3a984814c712ddefa59027a29d8e1b66a27bbe26" exitCode=0
Sep 06 00:58:23.379062 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.378702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr" event={"ID":"e8a9be9c-f71d-4b19-a072-db00ddd8a24c","Type":"ContainerDied","Data":"d7fc23a3e065f80efab672dc3a984814c712ddefa59027a29d8e1b66a27bbe26"}
Sep 06 00:58:23.380775 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.380754    2575 generic.go:356] "Generic (PLEG): container finished" podID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerID="64055b2402894c44d96761fee6415ffd0af21eef63b12fafa99078fa75db1984" exitCode=0
Sep 06 00:58:23.380906 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.380884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg" event={"ID":"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3","Type":"ContainerDied","Data":"64055b2402894c44d96761fee6415ffd0af21eef63b12fafa99078fa75db1984"}
Sep 06 00:58:23.382910 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.382841    2575 generic.go:356] "Generic (PLEG): container finished" podID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerID="19044c161394d2c63d61313e2e38d555e2859221170a1f6f7ce611f23fb3f3f4" exitCode=0
Sep 06 00:58:23.382997 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.382890    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v" event={"ID":"ae1cfc43-1182-42c6-9952-a6757fee0d1a","Type":"ContainerDied","Data":"19044c161394d2c63d61313e2e38d555e2859221170a1f6f7ce611f23fb3f3f4"}
Sep 06 00:58:23.385280 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.385259    2575 generic.go:356] "Generic (PLEG): container finished" podID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerID="1cc6b2fc4db7a4872fd36a51b4e8f9518e8e7f8e68852f30cf8f5b0567c1c816" exitCode=0
Sep 06 00:58:23.385361 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:23.385297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q" event={"ID":"90dca77a-dd41-45f4-9f77-308e66d999ba","Type":"ContainerDied","Data":"1cc6b2fc4db7a4872fd36a51b4e8f9518e8e7f8e68852f30cf8f5b0567c1c816"}
Sep 06 00:58:24.392594 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.392556    2575 generic.go:356] "Generic (PLEG): container finished" podID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerID="b7a5bd1f7b21d29e7e575de8cfcaa8aea8b48f945458fa3cc975a03297c55499" exitCode=0
Sep 06 00:58:24.393027 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.392640    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg" event={"ID":"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3","Type":"ContainerDied","Data":"b7a5bd1f7b21d29e7e575de8cfcaa8aea8b48f945458fa3cc975a03297c55499"}
Sep 06 00:58:24.394764 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.394735    2575 generic.go:356] "Generic (PLEG): container finished" podID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerID="f35fa2f41d33da087125108e38b898c2c934d4b2bca41f0ea5a4f9b36a6f849a" exitCode=0
Sep 06 00:58:24.394904 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.394817    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v" event={"ID":"ae1cfc43-1182-42c6-9952-a6757fee0d1a","Type":"ContainerDied","Data":"f35fa2f41d33da087125108e38b898c2c934d4b2bca41f0ea5a4f9b36a6f849a"}
Sep 06 00:58:24.397287 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.397247    2575 generic.go:356] "Generic (PLEG): container finished" podID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerID="60baaf4b5c45546dd36828a0ede16e44ebb8cfa7562cf1f62892b084cd7b7b43" exitCode=0
Sep 06 00:58:24.397631 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.397600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr" event={"ID":"e8a9be9c-f71d-4b19-a072-db00ddd8a24c","Type":"ContainerDied","Data":"60baaf4b5c45546dd36828a0ede16e44ebb8cfa7562cf1f62892b084cd7b7b43"}
Sep 06 00:58:24.532942 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.532916    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:24.633011 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.632967    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hctl\" (UniqueName: \"kubernetes.io/projected/90dca77a-dd41-45f4-9f77-308e66d999ba-kube-api-access-5hctl\") pod \"90dca77a-dd41-45f4-9f77-308e66d999ba\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") "
Sep 06 00:58:24.633011 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.633021    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-util\") pod \"90dca77a-dd41-45f4-9f77-308e66d999ba\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") "
Sep 06 00:58:24.633213 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.633069    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-bundle\") pod \"90dca77a-dd41-45f4-9f77-308e66d999ba\" (UID: \"90dca77a-dd41-45f4-9f77-308e66d999ba\") "
Sep 06 00:58:24.633548 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.633525    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-bundle" (OuterVolumeSpecName: "bundle") pod "90dca77a-dd41-45f4-9f77-308e66d999ba" (UID: "90dca77a-dd41-45f4-9f77-308e66d999ba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:24.635266 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.635243    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90dca77a-dd41-45f4-9f77-308e66d999ba-kube-api-access-5hctl" (OuterVolumeSpecName: "kube-api-access-5hctl") pod "90dca77a-dd41-45f4-9f77-308e66d999ba" (UID: "90dca77a-dd41-45f4-9f77-308e66d999ba"). InnerVolumeSpecName "kube-api-access-5hctl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:58:24.640285 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.640261    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-util" (OuterVolumeSpecName: "util") pod "90dca77a-dd41-45f4-9f77-308e66d999ba" (UID: "90dca77a-dd41-45f4-9f77-308e66d999ba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:24.733778 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.733731    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:24.733778 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.733773    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hctl\" (UniqueName: \"kubernetes.io/projected/90dca77a-dd41-45f4-9f77-308e66d999ba-kube-api-access-5hctl\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:24.733778 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:24.733787    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90dca77a-dd41-45f4-9f77-308e66d999ba-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.404270 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.404242    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q"
Sep 06 00:58:25.404685 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.404290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cj554q" event={"ID":"90dca77a-dd41-45f4-9f77-308e66d999ba","Type":"ContainerDied","Data":"3d82d52bf9250657c1c3567f5a72d2edff7c5013345da9f8d890744409b9bb38"}
Sep 06 00:58:25.404685 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.404336    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d82d52bf9250657c1c3567f5a72d2edff7c5013345da9f8d890744409b9bb38"
Sep 06 00:58:25.584486 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.584466    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:25.586591 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.586574    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:25.589639 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.589621    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:25.640504 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640482    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-bundle\") pod \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") "
Sep 06 00:58:25.640628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640528    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-util\") pod \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") "
Sep 06 00:58:25.640628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640566    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-util\") pod \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") "
Sep 06 00:58:25.640628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640588    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npj4j\" (UniqueName: \"kubernetes.io/projected/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-kube-api-access-npj4j\") pod \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") "
Sep 06 00:58:25.640628 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640605    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-bundle\") pod \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") "
Sep 06 00:58:25.640832 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640630    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-bundle\") pod \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\" (UID: \"e8a9be9c-f71d-4b19-a072-db00ddd8a24c\") "
Sep 06 00:58:25.640832 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640662    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-util\") pod \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") "
Sep 06 00:58:25.640832 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640696    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5ctn\" (UniqueName: \"kubernetes.io/projected/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-kube-api-access-q5ctn\") pod \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\" (UID: \"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3\") "
Sep 06 00:58:25.640832 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.640725    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpgcx\" (UniqueName: \"kubernetes.io/projected/ae1cfc43-1182-42c6-9952-a6757fee0d1a-kube-api-access-dpgcx\") pod \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\" (UID: \"ae1cfc43-1182-42c6-9952-a6757fee0d1a\") "
Sep 06 00:58:25.641306 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.641232    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-bundle" (OuterVolumeSpecName: "bundle") pod "ae1cfc43-1182-42c6-9952-a6757fee0d1a" (UID: "ae1cfc43-1182-42c6-9952-a6757fee0d1a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:25.642131 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.641275    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-bundle" (OuterVolumeSpecName: "bundle") pod "e8a9be9c-f71d-4b19-a072-db00ddd8a24c" (UID: "e8a9be9c-f71d-4b19-a072-db00ddd8a24c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:25.642321 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.642295    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-bundle" (OuterVolumeSpecName: "bundle") pod "7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" (UID: "7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:25.643120 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.643076    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-kube-api-access-npj4j" (OuterVolumeSpecName: "kube-api-access-npj4j") pod "e8a9be9c-f71d-4b19-a072-db00ddd8a24c" (UID: "e8a9be9c-f71d-4b19-a072-db00ddd8a24c"). InnerVolumeSpecName "kube-api-access-npj4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:58:25.643439 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.643416    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1cfc43-1182-42c6-9952-a6757fee0d1a-kube-api-access-dpgcx" (OuterVolumeSpecName: "kube-api-access-dpgcx") pod "ae1cfc43-1182-42c6-9952-a6757fee0d1a" (UID: "ae1cfc43-1182-42c6-9952-a6757fee0d1a"). InnerVolumeSpecName "kube-api-access-dpgcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:58:25.643618 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.643600    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-kube-api-access-q5ctn" (OuterVolumeSpecName: "kube-api-access-q5ctn") pod "7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" (UID: "7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3"). InnerVolumeSpecName "kube-api-access-q5ctn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:58:25.649322 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.649296    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-util" (OuterVolumeSpecName: "util") pod "ae1cfc43-1182-42c6-9952-a6757fee0d1a" (UID: "ae1cfc43-1182-42c6-9952-a6757fee0d1a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:25.649416 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.649386    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-util" (OuterVolumeSpecName: "util") pod "e8a9be9c-f71d-4b19-a072-db00ddd8a24c" (UID: "e8a9be9c-f71d-4b19-a072-db00ddd8a24c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:25.650741 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.650709    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-util" (OuterVolumeSpecName: "util") pod "7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" (UID: "7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 00:58:25.741483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741451    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpgcx\" (UniqueName: \"kubernetes.io/projected/ae1cfc43-1182-42c6-9952-a6757fee0d1a-kube-api-access-dpgcx\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741477    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741483 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741486    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741495    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741503    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npj4j\" (UniqueName: \"kubernetes.io/projected/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-kube-api-access-npj4j\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741511    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1cfc43-1182-42c6-9952-a6757fee0d1a-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741519    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8a9be9c-f71d-4b19-a072-db00ddd8a24c-bundle\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741526    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-util\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:25.741681 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:25.741534    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5ctn\" (UniqueName: \"kubernetes.io/projected/7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3-kube-api-access-q5ctn\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:58:26.410971 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.410946    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr"
Sep 06 00:58:26.411349 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.410942    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbnbtrr" event={"ID":"e8a9be9c-f71d-4b19-a072-db00ddd8a24c","Type":"ContainerDied","Data":"165149d3ad454f2e786319104f68f18ebeeb196e9ecdd0a3dca94c54d22b663d"}
Sep 06 00:58:26.411349 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.411096    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="165149d3ad454f2e786319104f68f18ebeeb196e9ecdd0a3dca94c54d22b663d"
Sep 06 00:58:26.413140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.413091    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg"
Sep 06 00:58:26.413140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.413094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jd8pg" event={"ID":"7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3","Type":"ContainerDied","Data":"a0061999b3b168d743265dd9bc642fe6bc32155030e54c8a2d58924fee6f2698"}
Sep 06 00:58:26.413140 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.413138    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0061999b3b168d743265dd9bc642fe6bc32155030e54c8a2d58924fee6f2698"
Sep 06 00:58:26.415432 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.415408    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v" event={"ID":"ae1cfc43-1182-42c6-9952-a6757fee0d1a","Type":"ContainerDied","Data":"4581807b8a88967545010b3de869538500a5a32735331ebb84506746a2216958"}
Sep 06 00:58:26.415432 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.415425    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0xv29v"
Sep 06 00:58:26.415576 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:26.415450    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4581807b8a88967545010b3de869538500a5a32735331ebb84506746a2216958"
Sep 06 00:58:30.663919 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:30.663877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:33.461280 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:33.461245    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:35.250589 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:35.250544    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:35.501451 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:35.501358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:35.650215 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:35.650183    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:36.264813 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:36.264780    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:36.701113 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:36.701082    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:37.792922 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:37.792889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:37.894394 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:37.894358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:39.503315 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:39.503265    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:39.879991 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:39.879902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:41.707230 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:41.707186    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:43.279528 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:43.279488    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:43.376025 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:43.375993    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:46.820729 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:46.820692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:47.069651 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:47.069619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:47.170491 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:47.170401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:47.269962 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:47.269928    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:50.837311 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:50.837266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:50.938307 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:50.938271    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:51.056966 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:51.056933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:51.172739 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:51.172651    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:53.982653 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:53.982613    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:59.739147 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:59.739112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:58:59.841518 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:58:59.841474    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:02.278902 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:02.278844    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:02.378916 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:02.378880    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:05.476300 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:05.476254    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:05.577709 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:05.577680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:09.983554 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:09.983517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:10.090497 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:10.090462    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:10.192547 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:10.192511    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:11.143315 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:11.143283    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:11.244492 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:11.244455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:18.689332 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:18.689288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:18.789917 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:18.789879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:26.163586 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:26.163537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:26.263236 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:26.263195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:37.968623 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:37.968529    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:38.065104 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.065067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:38.169066 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.169028    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:38.784435 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784401    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-77vp6"]
Sep 06 00:59:38.784719 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784706    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="util"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784720    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="util"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784731    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="extract"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784737    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="extract"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784745    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="util"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784750    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="util"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784756    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="extract"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784761    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="extract"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784767    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="extract"
Sep 06 00:59:38.784770 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784771    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="extract"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784781    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="util"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784785    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="util"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784791    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784796    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784808    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784812    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784817    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784822    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784827    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="extract"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784831    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="extract"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784838    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784842    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="pull"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784867    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="util"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784872    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="util"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784924    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef1a2cb-6898-48e7-95f0-a1f5caf2b0c3" containerName="extract"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784932    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a9be9c-f71d-4b19-a072-db00ddd8a24c" containerName="extract"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784938    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1cfc43-1182-42c6-9952-a6757fee0d1a" containerName="extract"
Sep 06 00:59:38.785087 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.784944    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="90dca77a-dd41-45f4-9f77-308e66d999ba" containerName="extract"
Sep 06 00:59:38.787539 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.787524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:38.789648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.789622    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 00:59:38.790363 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.790343    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pq2qr\""
Sep 06 00:59:38.790469 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.790384    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 00:59:38.796769 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.796748    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-77vp6"]
Sep 06 00:59:38.814524 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.814479    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:38.830382 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.830365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m59bj\" (UniqueName: \"kubernetes.io/projected/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe-kube-api-access-m59bj\") pod \"authorino-b54df94b4-77vp6\" (UID: \"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe\") " pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:38.909693 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.909650    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:38.910694 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.910674    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cz7ps"]
Sep 06 00:59:38.913686 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.913672    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 00:59:38.918697 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.918673    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cz7ps"]
Sep 06 00:59:38.931551 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.931524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m59bj\" (UniqueName: \"kubernetes.io/projected/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe-kube-api-access-m59bj\") pod \"authorino-b54df94b4-77vp6\" (UID: \"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe\") " pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:38.939460 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:38.939435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m59bj\" (UniqueName: \"kubernetes.io/projected/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe-kube-api-access-m59bj\") pod \"authorino-b54df94b4-77vp6\" (UID: \"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe\") " pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:39.032369 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.032342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg52k\" (UniqueName: \"kubernetes.io/projected/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8-kube-api-access-tg52k\") pod \"authorino-c548b5d7d-cz7ps\" (UID: \"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8\") " pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 00:59:39.096494 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.096413    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:39.134103 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.134068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg52k\" (UniqueName: \"kubernetes.io/projected/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8-kube-api-access-tg52k\") pod \"authorino-c548b5d7d-cz7ps\" (UID: \"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8\") " pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 00:59:39.144236 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.144215    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg52k\" (UniqueName: \"kubernetes.io/projected/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8-kube-api-access-tg52k\") pod \"authorino-c548b5d7d-cz7ps\" (UID: \"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8\") " pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 00:59:39.222955 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.222918    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 00:59:39.364459 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.364437    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-77vp6"]
Sep 06 00:59:39.366813 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:59:39.366791    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod648fdf1b_3048_4189_9a3c_ffe7ebc17bbe.slice/crio-7c45da55d7e6ca57bd0c6b16307a0818cfe823aa1c4f63f31628165a2f4d379c WatchSource:0}: Error finding container 7c45da55d7e6ca57bd0c6b16307a0818cfe823aa1c4f63f31628165a2f4d379c: Status 404 returned error can't find the container with id 7c45da55d7e6ca57bd0c6b16307a0818cfe823aa1c4f63f31628165a2f4d379c
Sep 06 00:59:39.496252 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.496230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cz7ps"]
Sep 06 00:59:39.497759 ip-10-0-129-5 kubenswrapper[2575]: W0906 00:59:39.497733    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0af0a9e6_55d3_4aae_a2a4_c1d44517c3f8.slice/crio-d274cb33f6c4cfacb1abafd3ea52fb51d4b643e520ba42b2419153eda07ccceb WatchSource:0}: Error finding container d274cb33f6c4cfacb1abafd3ea52fb51d4b643e520ba42b2419153eda07ccceb: Status 404 returned error can't find the container with id d274cb33f6c4cfacb1abafd3ea52fb51d4b643e520ba42b2419153eda07ccceb
Sep 06 00:59:39.780367 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.780328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cz7ps" event={"ID":"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8","Type":"ContainerStarted","Data":"d274cb33f6c4cfacb1abafd3ea52fb51d4b643e520ba42b2419153eda07ccceb"}
Sep 06 00:59:39.781510 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:39.781489    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-77vp6" event={"ID":"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe","Type":"ContainerStarted","Data":"7c45da55d7e6ca57bd0c6b16307a0818cfe823aa1c4f63f31628165a2f4d379c"}
Sep 06 00:59:42.802450 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:42.802417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cz7ps" event={"ID":"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8","Type":"ContainerStarted","Data":"b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096"}
Sep 06 00:59:42.803963 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:42.803937    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-77vp6" event={"ID":"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe","Type":"ContainerStarted","Data":"1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d"}
Sep 06 00:59:42.817189 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:42.817147    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-cz7ps" podStartSLOduration=1.990858569 podStartE2EDuration="4.817133262s" totalImagesPullingTime="2.826274693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:59:42.816267396 +0000 UTC m=+545.145626232" watchObservedRunningTime="2026-09-06 00:59:42.817133262 +0000 UTC m=+545.146492118"
Sep 06 00:59:42.832799 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:42.832760    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-77vp6" podStartSLOduration=1.8639605160000001 podStartE2EDuration="4.832744805s" totalImagesPullingTime="2.968784289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 00:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 00:59:42.831236964 +0000 UTC m=+545.160595801" watchObservedRunningTime="2026-09-06 00:59:42.832744805 +0000 UTC m=+545.162103640"
Sep 06 00:59:42.853883 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:42.853836    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-77vp6"]
Sep 06 00:59:44.814862 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:44.814813    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-77vp6" podUID="648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" containerName="authorino" containerID="cri-o://1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d" gracePeriod=30
Sep 06 00:59:45.061646 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.061623    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:45.185252 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.185166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m59bj\" (UniqueName: \"kubernetes.io/projected/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe-kube-api-access-m59bj\") pod \"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe\" (UID: \"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe\") "
Sep 06 00:59:45.187544 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.187519    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe-kube-api-access-m59bj" (OuterVolumeSpecName: "kube-api-access-m59bj") pod "648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" (UID: "648fdf1b-3048-4189-9a3c-ffe7ebc17bbe"). InnerVolumeSpecName "kube-api-access-m59bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 00:59:45.286632 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.286605    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m59bj\" (UniqueName: \"kubernetes.io/projected/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe-kube-api-access-m59bj\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 00:59:45.821648 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.821613    2575 generic.go:356] "Generic (PLEG): container finished" podID="648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" containerID="1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d" exitCode=0
Sep 06 00:59:45.822124 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.821654    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-77vp6"
Sep 06 00:59:45.822124 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.821704    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-77vp6" event={"ID":"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe","Type":"ContainerDied","Data":"1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d"}
Sep 06 00:59:45.822124 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.821750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-77vp6" event={"ID":"648fdf1b-3048-4189-9a3c-ffe7ebc17bbe","Type":"ContainerDied","Data":"7c45da55d7e6ca57bd0c6b16307a0818cfe823aa1c4f63f31628165a2f4d379c"}
Sep 06 00:59:45.822124 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.821774    2575 scope.go:117] "RemoveContainer" containerID="1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d"
Sep 06 00:59:45.830690 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.830673    2575 scope.go:117] "RemoveContainer" containerID="1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d"
Sep 06 00:59:45.830966 ip-10-0-129-5 kubenswrapper[2575]: E0906 00:59:45.830948    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d\": container with ID starting with 1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d not found: ID does not exist" containerID="1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d"
Sep 06 00:59:45.831026 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.830974    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d"} err="failed to get container status \"1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d\": rpc error: code = NotFound desc = could not find container \"1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d\": container with ID starting with 1e1e1c3c37dcb39bbb9403ed91b3f07b941f3970e193d55438035f0ad8be746d not found: ID does not exist"
Sep 06 00:59:45.841504 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.841485    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-77vp6"]
Sep 06 00:59:45.844261 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:45.844242    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-77vp6"]
Sep 06 00:59:46.201075 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:46.201047    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" path="/var/lib/kubelet/pods/648fdf1b-3048-4189-9a3c-ffe7ebc17bbe/volumes"
Sep 06 00:59:53.222686 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:53.222644    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:53.323877 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:53.323812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 00:59:53.706564 ip-10-0-129-5 kubenswrapper[2575]: I0906 00:59:53.706531    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:06.107614 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.107577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:06.270444 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.270406    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:06.780997 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.780964    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc9958cc4-h2qjm"]
Sep 06 01:00:06.781285 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.781273    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" containerName="authorino"
Sep 06 01:00:06.781328 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.781287    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" containerName="authorino"
Sep 06 01:00:06.781365 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.781342    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="648fdf1b-3048-4189-9a3c-ffe7ebc17bbe" containerName="authorino"
Sep 06 01:00:06.784076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.784061    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:06.793939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.793913    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc9958cc4-h2qjm"]
Sep 06 01:00:06.813336 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.813300    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:06.863418 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.863378    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc9958cc4-h2qjm"]
Sep 06 01:00:06.863649 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:00:06.863625    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-k8vr4], unattached volumes=[], failed to process volumes=[kube-api-access-k8vr4]: context canceled" pod="kuadrant-system/authorino-5dc9958cc4-h2qjm" podUID="62276fcb-4454-46ce-8d9a-a80631931edf"
Sep 06 01:00:06.928593 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.928562    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:06.935106 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.935082    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:06.963526 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:06.963473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8vr4\" (UniqueName: \"kubernetes.io/projected/62276fcb-4454-46ce-8d9a-a80631931edf-kube-api-access-k8vr4\") pod \"authorino-5dc9958cc4-h2qjm\" (UID: \"62276fcb-4454-46ce-8d9a-a80631931edf\") " pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:07.064502 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.064411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8vr4\" (UniqueName: \"kubernetes.io/projected/62276fcb-4454-46ce-8d9a-a80631931edf-kube-api-access-k8vr4\") pod \"authorino-5dc9958cc4-h2qjm\" (UID: \"62276fcb-4454-46ce-8d9a-a80631931edf\") " pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:07.074188 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.074164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8vr4\" (UniqueName: \"kubernetes.io/projected/62276fcb-4454-46ce-8d9a-a80631931edf-kube-api-access-k8vr4\") pod \"authorino-5dc9958cc4-h2qjm\" (UID: \"62276fcb-4454-46ce-8d9a-a80631931edf\") " pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:07.164742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.164704    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8vr4\" (UniqueName: \"kubernetes.io/projected/62276fcb-4454-46ce-8d9a-a80631931edf-kube-api-access-k8vr4\") pod \"62276fcb-4454-46ce-8d9a-a80631931edf\" (UID: \"62276fcb-4454-46ce-8d9a-a80631931edf\") "
Sep 06 01:00:07.166933 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.166906    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62276fcb-4454-46ce-8d9a-a80631931edf-kube-api-access-k8vr4" (OuterVolumeSpecName: "kube-api-access-k8vr4") pod "62276fcb-4454-46ce-8d9a-a80631931edf" (UID: "62276fcb-4454-46ce-8d9a-a80631931edf"). InnerVolumeSpecName "kube-api-access-k8vr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:00:07.265718 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.265686    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8vr4\" (UniqueName: \"kubernetes.io/projected/62276fcb-4454-46ce-8d9a-a80631931edf-kube-api-access-k8vr4\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:00:07.933254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.933217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc9958cc4-h2qjm"
Sep 06 01:00:07.961381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.961353    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc9958cc4-h2qjm"]
Sep 06 01:00:07.965646 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:07.965621    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc9958cc4-h2qjm"]
Sep 06 01:00:08.203109 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:08.203064    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62276fcb-4454-46ce-8d9a-a80631931edf" path="/var/lib/kubelet/pods/62276fcb-4454-46ce-8d9a-a80631931edf/volumes"
Sep 06 01:00:10.045447 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.045415    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cz7ps"]
Sep 06 01:00:10.045924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.045586    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-cz7ps" podUID="0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" containerName="authorino" containerID="cri-o://b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096" gracePeriod=30
Sep 06 01:00:10.290935 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.290910    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 01:00:10.392455 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.392370    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg52k\" (UniqueName: \"kubernetes.io/projected/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8-kube-api-access-tg52k\") pod \"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8\" (UID: \"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8\") "
Sep 06 01:00:10.394636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.394613    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8-kube-api-access-tg52k" (OuterVolumeSpecName: "kube-api-access-tg52k") pod "0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" (UID: "0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8"). InnerVolumeSpecName "kube-api-access-tg52k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:00:10.493475 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.493449    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg52k\" (UniqueName: \"kubernetes.io/projected/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8-kube-api-access-tg52k\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:00:10.949788 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.949750    2575 generic.go:356] "Generic (PLEG): container finished" podID="0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" containerID="b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096" exitCode=0
Sep 06 01:00:10.950007 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.949792    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cz7ps"
Sep 06 01:00:10.950007 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.949845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cz7ps" event={"ID":"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8","Type":"ContainerDied","Data":"b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096"}
Sep 06 01:00:10.950007 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.949919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cz7ps" event={"ID":"0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8","Type":"ContainerDied","Data":"d274cb33f6c4cfacb1abafd3ea52fb51d4b643e520ba42b2419153eda07ccceb"}
Sep 06 01:00:10.950007 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.949943    2575 scope.go:117] "RemoveContainer" containerID="b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096"
Sep 06 01:00:10.959060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.959043    2575 scope.go:117] "RemoveContainer" containerID="b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096"
Sep 06 01:00:10.959310 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:00:10.959292    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096\": container with ID starting with b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096 not found: ID does not exist" containerID="b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096"
Sep 06 01:00:10.959351 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.959318    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096"} err="failed to get container status \"b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096\": rpc error: code = NotFound desc = could not find container \"b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096\": container with ID starting with b04a1b8a18f3d5427bee3b4550a60e0886b4c2e9242bc74df78a55e21f6cd096 not found: ID does not exist"
Sep 06 01:00:10.971275 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.971250    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cz7ps"]
Sep 06 01:00:10.978256 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:10.978239    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cz7ps"]
Sep 06 01:00:12.201436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:12.201399    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" path="/var/lib/kubelet/pods/0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8/volumes"
Sep 06 01:00:14.840969 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:14.840942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:14.919664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:14.919622    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:14.980807 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:14.980781    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:15.429913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:15.429869    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:15.641407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:15.641375    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:19.579808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:19.579771    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:19.685061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:19.685026    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:35.295035 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:35.294993    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:35.350583 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:35.350546    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:38.144186 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:38.144154    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:00:38.144717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:38.144697    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:00:48.665693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:48.665653    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:50.616644 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.616604    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:50.716527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.716492    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:50.814328 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.814294    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:50.822125 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.822094    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-rjs67"]
Sep 06 01:00:50.822609 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.822593    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" containerName="authorino"
Sep 06 01:00:50.822655 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.822615    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" containerName="authorino"
Sep 06 01:00:50.822703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.822692    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af0a9e6-55d3-4aae-a2a4-c1d44517c3f8" containerName="authorino"
Sep 06 01:00:50.826098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.826078    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:50.828708 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.828691    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 06 01:00:50.828923 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.828908    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9dw5v\""
Sep 06 01:00:50.840223 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.840200    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-rjs67"]
Sep 06 01:00:50.918375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.918289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn4jg\" (UniqueName: \"kubernetes.io/projected/95231315-9b02-4114-9976-6af33cfc6af9-kube-api-access-mn4jg\") pod \"payload-processing-5b54b7fbb7-rjs67\" (UID: \"95231315-9b02-4114-9976-6af33cfc6af9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:50.918375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:50.918356    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95231315-9b02-4114-9976-6af33cfc6af9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-rjs67\" (UID: \"95231315-9b02-4114-9976-6af33cfc6af9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:51.019619 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.019589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn4jg\" (UniqueName: \"kubernetes.io/projected/95231315-9b02-4114-9976-6af33cfc6af9-kube-api-access-mn4jg\") pod \"payload-processing-5b54b7fbb7-rjs67\" (UID: \"95231315-9b02-4114-9976-6af33cfc6af9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:51.019815 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.019651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95231315-9b02-4114-9976-6af33cfc6af9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-rjs67\" (UID: \"95231315-9b02-4114-9976-6af33cfc6af9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:51.020423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.020396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95231315-9b02-4114-9976-6af33cfc6af9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-rjs67\" (UID: \"95231315-9b02-4114-9976-6af33cfc6af9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:51.030404 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.030380    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn4jg\" (UniqueName: \"kubernetes.io/projected/95231315-9b02-4114-9976-6af33cfc6af9-kube-api-access-mn4jg\") pod \"payload-processing-5b54b7fbb7-rjs67\" (UID: \"95231315-9b02-4114-9976-6af33cfc6af9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:51.104658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.104625    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:00:51.136330 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.136306    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:51.410839 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.410816    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-rjs67"]
Sep 06 01:00:51.412823 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:00:51.412798    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95231315_9b02_4114_9976_6af33cfc6af9.slice/crio-f48f44e9faabb5121e69bfa23ac8c38613db4f6f83744a1d330128e4b3630a30 WatchSource:0}: Error finding container f48f44e9faabb5121e69bfa23ac8c38613db4f6f83744a1d330128e4b3630a30: Status 404 returned error can't find the container with id f48f44e9faabb5121e69bfa23ac8c38613db4f6f83744a1d330128e4b3630a30
Sep 06 01:00:51.414665 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:51.414645    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 01:00:52.164584 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:52.164546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67" event={"ID":"95231315-9b02-4114-9976-6af33cfc6af9","Type":"ContainerStarted","Data":"f48f44e9faabb5121e69bfa23ac8c38613db4f6f83744a1d330128e4b3630a30"}
Sep 06 01:00:53.171005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:53.170974    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67" event={"ID":"95231315-9b02-4114-9976-6af33cfc6af9","Type":"ContainerStarted","Data":"1701d2625cb65a6ebd9f0535ca20657b41b6bd49f8bd47671e017383409bde8a"}
Sep 06 01:00:53.171286 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:53.171114    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:00:53.187941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:00:53.187900    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67" podStartSLOduration=1.499892177 podStartE2EDuration="3.187883531s" totalImagesPullingTime="1.687991354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:00:53.186422252 +0000 UTC m=+615.515781104" watchObservedRunningTime="2026-09-06 01:00:53.187883531 +0000 UTC m=+615.517242366"
Sep 06 01:01:01.737604 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:01.737505    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:01:01.836276 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:01.836238    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:01:04.177792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:04.177761    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-rjs67"
Sep 06 01:01:10.398436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:10.398400    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:01:16.893006 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:16.892969    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:01:41.688484 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:41.688452    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:01:52.554676 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.554644    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"]
Sep 06 01:01:52.558131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.558111    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.560990 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.560965    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 06 01:01:52.561687 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.561671    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 06 01:01:52.561796 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.561776    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 06 01:01:52.561977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.561957    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pg77g\""
Sep 06 01:01:52.569032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.569012    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"]
Sep 06 01:01:52.607004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.606964    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:01:52.626034 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.626012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.626168 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.626064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.626168 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.626087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67dba8e3-064a-44cf-83e7-675b837d6610-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.626168 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.626115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.626331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.626214    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbnhn\" (UniqueName: \"kubernetes.io/projected/67dba8e3-064a-44cf-83e7-675b837d6610-kube-api-access-nbnhn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.626331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.626296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727008    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67dba8e3-064a-44cf-83e7-675b837d6610-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbnhn\" (UniqueName: \"kubernetes.io/projected/67dba8e3-064a-44cf-83e7-675b837d6610-kube-api-access-nbnhn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727201    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727477 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727447    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727651 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.727732 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.727652    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.729704 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.729684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67dba8e3-064a-44cf-83e7-675b837d6610-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.729839 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.729822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67dba8e3-064a-44cf-83e7-675b837d6610-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.735290 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.735264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbnhn\" (UniqueName: \"kubernetes.io/projected/67dba8e3-064a-44cf-83e7-675b837d6610-kube-api-access-nbnhn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-sjd59\" (UID: \"67dba8e3-064a-44cf-83e7-675b837d6610\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:52.869098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:52.869016    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:01:53.159372 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:53.159350    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"]
Sep 06 01:01:53.161426 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:01:53.161399    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67dba8e3_064a_44cf_83e7_675b837d6610.slice/crio-6ff3dfd3643ac94c67ead3ed6fcd1acb06a8b87ad9a4fcc217b6da6a3b70cfa2 WatchSource:0}: Error finding container 6ff3dfd3643ac94c67ead3ed6fcd1acb06a8b87ad9a4fcc217b6da6a3b70cfa2: Status 404 returned error can't find the container with id 6ff3dfd3643ac94c67ead3ed6fcd1acb06a8b87ad9a4fcc217b6da6a3b70cfa2
Sep 06 01:01:53.474786 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:53.474741    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59" event={"ID":"67dba8e3-064a-44cf-83e7-675b837d6610","Type":"ContainerStarted","Data":"6ff3dfd3643ac94c67ead3ed6fcd1acb06a8b87ad9a4fcc217b6da6a3b70cfa2"}
Sep 06 01:01:54.481544 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:54.481509    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59" event={"ID":"67dba8e3-064a-44cf-83e7-675b837d6610","Type":"ContainerStarted","Data":"5008f45471ed4f5ac969b4d3056cbb929e02bcadf52f6a4520d7756dd812af05"}
Sep 06 01:01:54.500117 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:01:54.500068    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59" podStartSLOduration=1.560650417 podStartE2EDuration="2.50005336s" totalImagesPullingTime="939.402943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:01:54.499190112 +0000 UTC m=+676.828548947" watchObservedRunningTime="2026-09-06 01:01:54.50005336 +0000 UTC m=+676.829412196"
Sep 06 01:02:00.989588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:00.989546    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:02:02.869639 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:02.869598    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:02:02.870085 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:02.869677    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:02:02.882341 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:02.882318    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:02:03.535760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:03.535733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-sjd59"
Sep 06 01:02:08.269791 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:08.269759    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:02:10.715522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:10.715488    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:02:19.237139 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:19.237104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:02:26.864508 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.864467    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"]
Sep 06 01:02:26.868307 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.868283    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:26.870487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.870464    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 06 01:02:26.877618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.877597    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"]
Sep 06 01:02:26.885490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.885465    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:02:26.912771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.912747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:26.912903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.912777    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:26.912903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.912799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg58l\" (UniqueName: \"kubernetes.io/projected/09b0837d-b499-479b-82bc-b0e4dda0b55f-kube-api-access-tg58l\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:26.912991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.912916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:26.912991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.912970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09b0837d-b499-479b-82bc-b0e4dda0b55f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:26.913061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:26.913008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014217 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014354 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014354 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014248    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg58l\" (UniqueName: \"kubernetes.io/projected/09b0837d-b499-479b-82bc-b0e4dda0b55f-kube-api-access-tg58l\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014354 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014354 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09b0837d-b499-479b-82bc-b0e4dda0b55f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014361    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014662    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.014730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.014707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.016700 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.016673    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09b0837d-b499-479b-82bc-b0e4dda0b55f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.017040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.017022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09b0837d-b499-479b-82bc-b0e4dda0b55f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.021785 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.021756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg58l\" (UniqueName: \"kubernetes.io/projected/09b0837d-b499-479b-82bc-b0e4dda0b55f-kube-api-access-tg58l\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-95tbs\" (UID: \"09b0837d-b499-479b-82bc-b0e4dda0b55f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.177943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.177829    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:27.450770 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.450746    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"]
Sep 06 01:02:27.452457 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:02:27.452429    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09b0837d_b499_479b_82bc_b0e4dda0b55f.slice/crio-076aad14395e30277eaaee1f697a087c96a9dd70b2aa4c5e0858889b40d51fe0 WatchSource:0}: Error finding container 076aad14395e30277eaaee1f697a087c96a9dd70b2aa4c5e0858889b40d51fe0: Status 404 returned error can't find the container with id 076aad14395e30277eaaee1f697a087c96a9dd70b2aa4c5e0858889b40d51fe0
Sep 06 01:02:27.645175 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:27.645148    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs" event={"ID":"09b0837d-b499-479b-82bc-b0e4dda0b55f","Type":"ContainerStarted","Data":"076aad14395e30277eaaee1f697a087c96a9dd70b2aa4c5e0858889b40d51fe0"}
Sep 06 01:02:28.651254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:28.651218    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs" event={"ID":"09b0837d-b499-479b-82bc-b0e4dda0b55f","Type":"ContainerStarted","Data":"1cfc916c95d908e98232d6dcca8c77e1071e27c171ce55c347fcc48d9dd65c06"}
Sep 06 01:02:28.670538 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:28.670491    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs" podStartSLOduration=2.499455592 podStartE2EDuration="2.670474598s" totalImagesPullingTime="171.019006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:02:28.668736583 +0000 UTC m=+710.998095420" watchObservedRunningTime="2026-09-06 01:02:28.670474598 +0000 UTC m=+710.999833434"
Sep 06 01:02:37.178723 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:37.178685    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:37.179240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:37.178737    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:37.191787 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:37.191760    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:02:37.708745 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:02:37.708713    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-95tbs"
Sep 06 01:03:57.875896 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.875865    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59567f46df-gg242"]
Sep 06 01:03:57.879103 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.879088    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:57.881929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.881904    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 06 01:03:57.881929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.881907    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pq2qr\""
Sep 06 01:03:57.882593 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.882576    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 06 01:03:57.882665 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.882577    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 06 01:03:57.886636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.886611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59567f46df-gg242"]
Sep 06 01:03:57.992642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.992609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5c69e02-23ab-4640-aaea-43146c344437-tls-cert\") pod \"authorino-59567f46df-gg242\" (UID: \"c5c69e02-23ab-4640-aaea-43146c344437\") " pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:57.992642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:57.992646    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4g8g\" (UniqueName: \"kubernetes.io/projected/c5c69e02-23ab-4640-aaea-43146c344437-kube-api-access-n4g8g\") pod \"authorino-59567f46df-gg242\" (UID: \"c5c69e02-23ab-4640-aaea-43146c344437\") " pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:58.093069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:58.093035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5c69e02-23ab-4640-aaea-43146c344437-tls-cert\") pod \"authorino-59567f46df-gg242\" (UID: \"c5c69e02-23ab-4640-aaea-43146c344437\") " pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:58.093069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:58.093072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4g8g\" (UniqueName: \"kubernetes.io/projected/c5c69e02-23ab-4640-aaea-43146c344437-kube-api-access-n4g8g\") pod \"authorino-59567f46df-gg242\" (UID: \"c5c69e02-23ab-4640-aaea-43146c344437\") " pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:58.095749 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:58.095712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c5c69e02-23ab-4640-aaea-43146c344437-tls-cert\") pod \"authorino-59567f46df-gg242\" (UID: \"c5c69e02-23ab-4640-aaea-43146c344437\") " pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:58.103716 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:58.103689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4g8g\" (UniqueName: \"kubernetes.io/projected/c5c69e02-23ab-4640-aaea-43146c344437-kube-api-access-n4g8g\") pod \"authorino-59567f46df-gg242\" (UID: \"c5c69e02-23ab-4640-aaea-43146c344437\") " pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:58.190673 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:58.190606    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59567f46df-gg242"
Sep 06 01:03:58.463371 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:58.463345    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59567f46df-gg242"]
Sep 06 01:03:58.465260 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:03:58.465228    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5c69e02_23ab_4640_aaea_43146c344437.slice/crio-a975c62660baaa83c08ab7eb0931beaa297ef0143104139aa63b7d31e82b8268 WatchSource:0}: Error finding container a975c62660baaa83c08ab7eb0931beaa297ef0143104139aa63b7d31e82b8268: Status 404 returned error can't find the container with id a975c62660baaa83c08ab7eb0931beaa297ef0143104139aa63b7d31e82b8268
Sep 06 01:03:59.115992 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:03:59.115964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59567f46df-gg242" event={"ID":"c5c69e02-23ab-4640-aaea-43146c344437","Type":"ContainerStarted","Data":"a975c62660baaa83c08ab7eb0931beaa297ef0143104139aa63b7d31e82b8268"}
Sep 06 01:04:00.123924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:00.123884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59567f46df-gg242" event={"ID":"c5c69e02-23ab-4640-aaea-43146c344437","Type":"ContainerStarted","Data":"b6cdde20966be957c0b0fb4092d6634cbe262171ee4d0e14556b6686e194850c"}
Sep 06 01:04:00.150895 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:00.150813    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59567f46df-gg242" podStartSLOduration=2.529818588 podStartE2EDuration="3.150790204s" totalImagesPullingTime="620.971616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:04:00.14851227 +0000 UTC m=+802.477871103" watchObservedRunningTime="2026-09-06 01:04:00.150790204 +0000 UTC m=+802.480149033"
Sep 06 01:04:00.983756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:00.983721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:01.099996 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:01.099969    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:02.156947 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:02.156911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:02.278881 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:02.278823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:25.900519 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:25.900473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:26.003418 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:26.003386    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:26.258391 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:26.258343    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:26.600386 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:26.600296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:27.385352 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.385294    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:27.488161 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.488116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:27.865324 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.865277    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:27.876124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.876094    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7790-139700311596208"]
Sep 06 01:04:27.879720 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.879698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:27.881991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.881974    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tmqsk\""
Sep 06 01:04:27.887004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:27.886975    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7790-139700311596208"]
Sep 06 01:04:28.051793 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:28.051762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psqnx\" (UniqueName: \"kubernetes.io/projected/92279934-603e-4ad5-9371-aee3075193a5-kube-api-access-psqnx\") pod \"test-curl-7790-139700311596208\" (UID: \"92279934-603e-4ad5-9371-aee3075193a5\") " pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:28.153265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:28.153187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psqnx\" (UniqueName: \"kubernetes.io/projected/92279934-603e-4ad5-9371-aee3075193a5-kube-api-access-psqnx\") pod \"test-curl-7790-139700311596208\" (UID: \"92279934-603e-4ad5-9371-aee3075193a5\") " pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:28.162193 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:28.162168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psqnx\" (UniqueName: \"kubernetes.io/projected/92279934-603e-4ad5-9371-aee3075193a5-kube-api-access-psqnx\") pod \"test-curl-7790-139700311596208\" (UID: \"92279934-603e-4ad5-9371-aee3075193a5\") " pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:28.190029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:28.190007    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:28.475446 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:28.475425    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7790-139700311596208"]
Sep 06 01:04:28.477721 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:28.477695    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92279934_603e_4ad5_9371_aee3075193a5.slice/crio-8c17b18f09c81c37c21748344555695f62c4c716cbc28e9de6c86e514b66bba0 WatchSource:0}: Error finding container 8c17b18f09c81c37c21748344555695f62c4c716cbc28e9de6c86e514b66bba0: Status 404 returned error can't find the container with id 8c17b18f09c81c37c21748344555695f62c4c716cbc28e9de6c86e514b66bba0
Sep 06 01:04:29.274633 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:29.274598    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7790-139700311596208" event={"ID":"92279934-603e-4ad5-9371-aee3075193a5","Type":"ContainerStarted","Data":"8c17b18f09c81c37c21748344555695f62c4c716cbc28e9de6c86e514b66bba0"}
Sep 06 01:04:29.307677 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:29.307648    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:29.406078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:29.406044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:30.578754 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:30.578715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:30.662582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:30.662541    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:31.080058 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:31.080035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:31.177089 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:31.177056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:31.287503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:31.287425    2575 generic.go:356] "Generic (PLEG): container finished" podID="92279934-603e-4ad5-9371-aee3075193a5" containerID="d7a653f80a88cb9de390fa6b3474af743f3e746a5236d1e246546200d11b7c3d" exitCode=0
Sep 06 01:04:31.287634 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:31.287515    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7790-139700311596208" event={"ID":"92279934-603e-4ad5-9371-aee3075193a5","Type":"ContainerDied","Data":"d7a653f80a88cb9de390fa6b3474af743f3e746a5236d1e246546200d11b7c3d"}
Sep 06 01:04:31.649951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:31.649873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:32.418722 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.418695    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:32.464847 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.464822    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7790-139700311596208_92279934-603e-4ad5-9371-aee3075193a5/test-curl-7790-139700311596208/0.log"
Sep 06 01:04:32.492084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.492056    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psqnx\" (UniqueName: \"kubernetes.io/projected/92279934-603e-4ad5-9371-aee3075193a5-kube-api-access-psqnx\") pod \"92279934-603e-4ad5-9371-aee3075193a5\" (UID: \"92279934-603e-4ad5-9371-aee3075193a5\") "
Sep 06 01:04:32.494261 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.494235    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92279934-603e-4ad5-9371-aee3075193a5-kube-api-access-psqnx" (OuterVolumeSpecName: "kube-api-access-psqnx") pod "92279934-603e-4ad5-9371-aee3075193a5" (UID: "92279934-603e-4ad5-9371-aee3075193a5"). InnerVolumeSpecName "kube-api-access-psqnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:04:32.503450 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.503424    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7790-139700311596208"]
Sep 06 01:04:32.506629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.506607    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7790-139700311596208"]
Sep 06 01:04:32.593511 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.593437    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psqnx\" (UniqueName: \"kubernetes.io/projected/92279934-603e-4ad5-9371-aee3075193a5-kube-api-access-psqnx\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:04:32.629944 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.629918    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7790-139700311595952"]
Sep 06 01:04:32.630418 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.630400    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92279934-603e-4ad5-9371-aee3075193a5" containerName="test-curl-7790-139700311596208"
Sep 06 01:04:32.630418 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.630419    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="92279934-603e-4ad5-9371-aee3075193a5" containerName="test-curl-7790-139700311596208"
Sep 06 01:04:32.630535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.630493    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="92279934-603e-4ad5-9371-aee3075193a5" containerName="test-curl-7790-139700311596208"
Sep 06 01:04:32.632969 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.632955    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:32.655151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.655132    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7790-139700311595952"]
Sep 06 01:04:32.693720 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.693698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq48q\" (UniqueName: \"kubernetes.io/projected/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f-kube-api-access-tq48q\") pod \"test-curl-7790-139700311595952\" (UID: \"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f\") " pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:32.794968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.794938    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq48q\" (UniqueName: \"kubernetes.io/projected/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f-kube-api-access-tq48q\") pod \"test-curl-7790-139700311595952\" (UID: \"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f\") " pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:32.805171 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.805147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq48q\" (UniqueName: \"kubernetes.io/projected/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f-kube-api-access-tq48q\") pod \"test-curl-7790-139700311595952\" (UID: \"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f\") " pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:32.941912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:32.941834    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:33.213588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:33.213563    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7790-139700311595952"]
Sep 06 01:04:33.215226 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:33.215200    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab89fc8b_9ede_4c51_8e3a_5dfa88993b6f.slice/crio-f748cc17e2bb46c45202e6535fb8aaabb4c22093b1d1ca75ca2f8811f1164f37 WatchSource:0}: Error finding container f748cc17e2bb46c45202e6535fb8aaabb4c22093b1d1ca75ca2f8811f1164f37: Status 404 returned error can't find the container with id f748cc17e2bb46c45202e6535fb8aaabb4c22093b1d1ca75ca2f8811f1164f37
Sep 06 01:04:33.298915 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:33.298884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7790-139700311595952" event={"ID":"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f","Type":"ContainerStarted","Data":"f748cc17e2bb46c45202e6535fb8aaabb4c22093b1d1ca75ca2f8811f1164f37"}
Sep 06 01:04:33.300158 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:33.300135    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c17b18f09c81c37c21748344555695f62c4c716cbc28e9de6c86e514b66bba0"
Sep 06 01:04:33.300265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:33.300173    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:33.310236 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:04:33.310209    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7790-139700311596208\" is forbidden: User \"system:node:ip-10-0-129-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-5.ec2.internal' and this object" podUID="92279934-603e-4ad5-9371-aee3075193a5" pod="opendatahub/test-curl-7790-139700311596208"
Sep 06 01:04:34.201660 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:34.201629    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92279934-603e-4ad5-9371-aee3075193a5" path="/var/lib/kubelet/pods/92279934-603e-4ad5-9371-aee3075193a5/volumes"
Sep 06 01:04:34.307013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:34.306979    2575 generic.go:356] "Generic (PLEG): container finished" podID="ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f" containerID="2ef16d5ff15f856ebc897e9062deea1025ba0679fa63dc2a3277ade25a1f7eed" exitCode=0
Sep 06 01:04:34.307173 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:34.307063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7790-139700311595952" event={"ID":"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f","Type":"ContainerDied","Data":"2ef16d5ff15f856ebc897e9062deea1025ba0679fa63dc2a3277ade25a1f7eed"}
Sep 06 01:04:35.342184 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.342153    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:35.437267 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.437243    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:35.493893 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.493872    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7790-139700311595952_ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f/test-curl-7790-139700311595952/0.log"
Sep 06 01:04:35.510505 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.510482    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:35.519314 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.519283    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq48q\" (UniqueName: \"kubernetes.io/projected/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f-kube-api-access-tq48q\") pod \"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f\" (UID: \"ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f\") "
Sep 06 01:04:35.521395 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.521363    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f-kube-api-access-tq48q" (OuterVolumeSpecName: "kube-api-access-tq48q") pod "ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f" (UID: "ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f"). InnerVolumeSpecName "kube-api-access-tq48q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:04:35.544674 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.544648    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7790-139700311595952"]
Sep 06 01:04:35.562944 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.562886    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7790-139700311595952"]
Sep 06 01:04:35.620951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.620923    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq48q\" (UniqueName: \"kubernetes.io/projected/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f-kube-api-access-tq48q\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:04:35.645871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.645822    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:35.981265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.981235    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"]
Sep 06 01:04:35.981589 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.981578    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f" containerName="test-curl-7790-139700311595952"
Sep 06 01:04:35.981636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.981591    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f" containerName="test-curl-7790-139700311595952"
Sep 06 01:04:35.981675 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.981659    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f" containerName="test-curl-7790-139700311595952"
Sep 06 01:04:35.984394 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.984377    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:35.989406 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:35.989388    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d3f8b122-openshift-default-dockercfg-klllz\""
Sep 06 01:04:36.024311 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024278    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-data\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024311 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024311    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-token\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024370    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-envoy\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024410    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-podinfo\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-socket\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024475    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istiod-ca-cert\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkcpm\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-kube-api-access-gkcpm\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-credential-socket\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.024844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.024696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-certs\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.074509 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.074476    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"]
Sep 06 01:04:36.105450 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.105420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:36.125773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.125738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-certs\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.125938 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.125788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-data\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.125938 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.125908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-token\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.125951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-envoy\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.125989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-podinfo\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.126028    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-socket\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.126056    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istiod-ca-cert\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.126084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkcpm\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-kube-api-access-gkcpm\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.126122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-credential-socket\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.126202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-data\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.126271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.126249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-certs\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.134713 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.127368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-credential-socket\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.134713 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.127396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-socket\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.134713 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.127672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istiod-ca-cert\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.136997 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.136976    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-envoy\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.137181 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.137160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-podinfo\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.141409 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.141384    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkcpm\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-kube-api-access-gkcpm\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.144245 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.144223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-token\") pod \"e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") " pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.202683 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.202639    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f" path="/var/lib/kubelet/pods/ab89fc8b-9ede-4c51-8e3a-5dfa88993b6f/volumes"
Sep 06 01:04:36.240269 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.240189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:36.295388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.295352    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:36.322375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.322353    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7790-139700311595952"
Sep 06 01:04:36.322508 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.322351    2575 scope.go:117] "RemoveContainer" containerID="2ef16d5ff15f856ebc897e9062deea1025ba0679fa63dc2a3277ade25a1f7eed"
Sep 06 01:04:36.357363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.354149    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:36.566309 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.566223    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"]
Sep 06 01:04:36.571026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.570996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.579424 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.579276    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-639343-openshift-default-dockercfg-nmnwp\""
Sep 06 01:04:36.593503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.593471    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"]
Sep 06 01:04:36.599336 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.599314    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"]
Sep 06 01:04:36.602548 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:36.602522    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2a85ab_8391_4f1f_8dfa_2101cc328435.slice/crio-46c16e60a4d44fbc5ad35469bb1b88c4990185a99d5bfd25ffec9fd8bdee68b5 WatchSource:0}: Error finding container 46c16e60a4d44fbc5ad35469bb1b88c4990185a99d5bfd25ffec9fd8bdee68b5: Status 404 returned error can't find the container with id 46c16e60a4d44fbc5ad35469bb1b88c4990185a99d5bfd25ffec9fd8bdee68b5
Sep 06 01:04:36.605572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.605539    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:04:36.605742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.605723    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:04:36.605844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.605804    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:04:36.631600 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-token\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.631698 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-certs\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.631698 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-data\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.631805 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-socket\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.631893 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631878    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-envoy\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.631937 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-podinfo\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.631979 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.631965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istiod-ca-cert\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.632034 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.632021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-credential-socket\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.632077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.632066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfw5d\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-kube-api-access-gfw5d\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.690162 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.690133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:36.732631 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732595    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-envoy\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.732783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732643    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-podinfo\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.732783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istiod-ca-cert\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.732783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-credential-socket\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.732783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732761    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfw5d\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-kube-api-access-gfw5d\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732792    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-token\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732835    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-certs\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-data\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.732929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-socket\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733993 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.733431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-certs\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733993 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.733514    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-credential-socket\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.733993 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.733661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-socket\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.734188 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.734158    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istiod-ca-cert\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.734250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.734234    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-data\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.735654 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.735600    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-envoy\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.735999 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.735977    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-podinfo\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.747055 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.747032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-token\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.754109 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.754080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfw5d\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-kube-api-access-gfw5d\") pod \"e2e-autores-639343-openshift-default-684844c85d-b8f2q\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") " pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:36.876777 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.876652    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:36.884648 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:36.884411    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:37.236292 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.236263    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"]
Sep 06 01:04:37.239823 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:37.239801    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a8e0eb_fdf5_4044_8f68_61f1f0791496.slice/crio-e669c327b26bb2548010d16965cefcca0c505b8fcd68374092a45448d0e47c85 WatchSource:0}: Error finding container e669c327b26bb2548010d16965cefcca0c505b8fcd68374092a45448d0e47c85: Status 404 returned error can't find the container with id e669c327b26bb2548010d16965cefcca0c505b8fcd68374092a45448d0e47c85
Sep 06 01:04:37.242125 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.242097    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:04:37.242195 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.242161    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:04:37.242195 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.242190    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:04:37.331168 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.331137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" event={"ID":"8f2a85ab-8391-4f1f-8dfa-2101cc328435","Type":"ContainerStarted","Data":"96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"}
Sep 06 01:04:37.331330 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.331199    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" event={"ID":"8f2a85ab-8391-4f1f-8dfa-2101cc328435","Type":"ContainerStarted","Data":"46c16e60a4d44fbc5ad35469bb1b88c4990185a99d5bfd25ffec9fd8bdee68b5"}
Sep 06 01:04:37.333991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.333955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" event={"ID":"d7a8e0eb-fdf5-4044-8f68-61f1f0791496","Type":"ContainerStarted","Data":"b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"}
Sep 06 01:04:37.334102 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.334013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" event={"ID":"d7a8e0eb-fdf5-4044-8f68-61f1f0791496","Type":"ContainerStarted","Data":"e669c327b26bb2548010d16965cefcca0c505b8fcd68374092a45448d0e47c85"}
Sep 06 01:04:37.436028 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.435977    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podStartSLOduration=2.4359600869999998 podStartE2EDuration="2.435960087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:04:37.434227128 +0000 UTC m=+839.763586002" watchObservedRunningTime="2026-09-06 01:04:37.435960087 +0000 UTC m=+839.765318923"
Sep 06 01:04:37.476749 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.476702    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podStartSLOduration=1.476681507 podStartE2EDuration="1.476681507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:04:37.476601288 +0000 UTC m=+839.805960141" watchObservedRunningTime="2026-09-06 01:04:37.476681507 +0000 UTC m=+839.806040348"
Sep 06 01:04:37.885550 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:37.885516    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:38.296289 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:38.296259    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:38.301192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:38.301170    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:38.339285 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:38.339242    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:38.340040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:38.340023    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:04:38.890570 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:38.890543    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:38.951029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:38.950990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:39.062489 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:39.062460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:39.344216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:39.344181    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:39.345090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:39.345070    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:04:39.535061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:39.535024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:39.664006 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:39.663928    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:40.098069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:40.098030    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:40.258828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:40.258796    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:40.592544 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:40.592515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:40.845004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:40.844933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:40.951948 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:40.951909    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:41.046840 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.046814    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"]
Sep 06 01:04:41.051903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.051885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.055244 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.055216    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d3f8b122-dockercfg-r8pbb\""
Sep 06 01:04:41.055344 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.055224    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d3f8b122\""
Sep 06 01:04:41.062693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.062669    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"]
Sep 06 01:04:41.072050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.072029    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd9qz\" (UniqueName: \"kubernetes.io/projected/5435d748-9689-4c45-a6ba-791f3bdda286-kube-api-access-bd9qz\") pod \"payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.072127 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.072069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5435d748-9689-4c45-a6ba-791f3bdda286-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.173098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.173022    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd9qz\" (UniqueName: \"kubernetes.io/projected/5435d748-9689-4c45-a6ba-791f3bdda286-kube-api-access-bd9qz\") pod \"payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.173098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.173068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5435d748-9689-4c45-a6ba-791f3bdda286-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.173712 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.173691    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5435d748-9689-4c45-a6ba-791f3bdda286-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.183347 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.183326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd9qz\" (UniqueName: \"kubernetes.io/projected/5435d748-9689-4c45-a6ba-791f3bdda286-kube-api-access-bd9qz\") pod \"payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.362381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.362348    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:41.498802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.498769    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:41.661488 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.661455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:41.685301 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.685275    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"]
Sep 06 01:04:41.685387 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:41.685291    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5435d748_9689_4c45_a6ba_791f3bdda286.slice/crio-f12d8a648aed122f4aacecf1b0f78fc506b7608209216279ef55645e17798645 WatchSource:0}: Error finding container f12d8a648aed122f4aacecf1b0f78fc506b7608209216279ef55645e17798645: Status 404 returned error can't find the container with id f12d8a648aed122f4aacecf1b0f78fc506b7608209216279ef55645e17798645
Sep 06 01:04:41.886384 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:41.886349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:42.368518 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.368485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2" event={"ID":"5435d748-9689-4c45-a6ba-791f3bdda286","Type":"ContainerStarted","Data":"a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b"}
Sep 06 01:04:42.368939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.368674    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:42.368939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.368706    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2" event={"ID":"5435d748-9689-4c45-a6ba-791f3bdda286","Type":"ContainerStarted","Data":"f12d8a648aed122f4aacecf1b0f78fc506b7608209216279ef55645e17798645"}
Sep 06 01:04:42.573551 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.573518    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:42.716971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.716935    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:42.773846 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.773796    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2" podStartSLOduration=1.773777343 podStartE2EDuration="1.773777343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:04:42.404503641 +0000 UTC m=+844.733862502" watchObservedRunningTime="2026-09-06 01:04:42.773777343 +0000 UTC m=+845.103136182"
Sep 06 01:04:42.775277 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.775257    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"]
Sep 06 01:04:42.778731 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.778716    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:42.783427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.783384    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-639343-dockercfg-r2plv\""
Sep 06 01:04:42.783657 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.783519    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-639343\""
Sep 06 01:04:42.802486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.802462    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"]
Sep 06 01:04:42.814689 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.814663    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"]
Sep 06 01:04:42.818073 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.818055    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.847968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.847947    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"]
Sep 06 01:04:42.886905 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.886880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c7d98b3-d026-4467-88fa-f4af9c08971e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:42.887021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.886931    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0928735-2e49-4844-823e-fabc3484c85a-plugins-config-volume\") pod \"payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") " pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.887021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.886951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hrv\" (UniqueName: \"kubernetes.io/projected/0c7d98b3-d026-4467-88fa-f4af9c08971e-kube-api-access-q4hrv\") pod \"payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:42.887125 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.887022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qlgq\" (UniqueName: \"kubernetes.io/projected/e0928735-2e49-4844-823e-fabc3484c85a-kube-api-access-6qlgq\") pod \"payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") " pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.988294 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.988214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0928735-2e49-4844-823e-fabc3484c85a-plugins-config-volume\") pod \"payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") " pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.988294 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.988255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hrv\" (UniqueName: \"kubernetes.io/projected/0c7d98b3-d026-4467-88fa-f4af9c08971e-kube-api-access-q4hrv\") pod \"payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:42.988471 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.988299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qlgq\" (UniqueName: \"kubernetes.io/projected/e0928735-2e49-4844-823e-fabc3484c85a-kube-api-access-6qlgq\") pod \"payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") " pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.988471 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.988350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c7d98b3-d026-4467-88fa-f4af9c08971e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:42.989021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.988998    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0928735-2e49-4844-823e-fabc3484c85a-plugins-config-volume\") pod \"payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") " pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.989070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.989053    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c7d98b3-d026-4467-88fa-f4af9c08971e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:42.998675 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.998644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qlgq\" (UniqueName: \"kubernetes.io/projected/e0928735-2e49-4844-823e-fabc3484c85a-kube-api-access-6qlgq\") pod \"payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") " pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:42.998675 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:42.998659    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hrv\" (UniqueName: \"kubernetes.io/projected/0c7d98b3-d026-4467-88fa-f4af9c08971e-kube-api-access-q4hrv\") pod \"payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:43.089372 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:43.089333    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:43.127952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:43.127920    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:43.302706 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:43.302637    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:43.375942 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:43.375914    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"]
Sep 06 01:04:43.378613 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:43.378586    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7d98b3_d026_4467_88fa_f4af9c08971e.slice/crio-c357c61c989a7d09d9547dcbc0e8c2b958f3cb3e63963e8d137fe8ece37c9d7d WatchSource:0}: Error finding container c357c61c989a7d09d9547dcbc0e8c2b958f3cb3e63963e8d137fe8ece37c9d7d: Status 404 returned error can't find the container with id c357c61c989a7d09d9547dcbc0e8c2b958f3cb3e63963e8d137fe8ece37c9d7d
Sep 06 01:04:43.425637 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:43.425613    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"]
Sep 06 01:04:43.427498 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:04:43.427476    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0928735_2e49_4844_823e_fabc3484c85a.slice/crio-bcf847e747611ecfa99062d5b139cbcbe14033e331442561886bc9a626799b63 WatchSource:0}: Error finding container bcf847e747611ecfa99062d5b139cbcbe14033e331442561886bc9a626799b63: Status 404 returned error can't find the container with id bcf847e747611ecfa99062d5b139cbcbe14033e331442561886bc9a626799b63
Sep 06 01:04:44.249765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.249733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:44.382613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.382582    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c" event={"ID":"0c7d98b3-d026-4467-88fa-f4af9c08971e","Type":"ContainerStarted","Data":"623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff"}
Sep 06 01:04:44.383055 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.382632    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c" event={"ID":"0c7d98b3-d026-4467-88fa-f4af9c08971e","Type":"ContainerStarted","Data":"c357c61c989a7d09d9547dcbc0e8c2b958f3cb3e63963e8d137fe8ece37c9d7d"}
Sep 06 01:04:44.383055 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.382663    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:44.384359 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.384321    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b" event={"ID":"e0928735-2e49-4844-823e-fabc3484c85a","Type":"ContainerStarted","Data":"244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410"}
Sep 06 01:04:44.384503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.384371    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b" event={"ID":"e0928735-2e49-4844-823e-fabc3484c85a","Type":"ContainerStarted","Data":"bcf847e747611ecfa99062d5b139cbcbe14033e331442561886bc9a626799b63"}
Sep 06 01:04:44.384503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.384441    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:44.402976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.402930    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c" podStartSLOduration=2.4029176 podStartE2EDuration="2.4029176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:04:44.399904989 +0000 UTC m=+846.729263845" watchObservedRunningTime="2026-09-06 01:04:44.4029176 +0000 UTC m=+846.732276471"
Sep 06 01:04:44.419931 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.419876    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b" podStartSLOduration=2.419835305 podStartE2EDuration="2.419835305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:04:44.418713647 +0000 UTC m=+846.748072516" watchObservedRunningTime="2026-09-06 01:04:44.419835305 +0000 UTC m=+846.749194142"
Sep 06 01:04:44.890047 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:44.890013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:45.065994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:45.065939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:45.208456 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:45.208420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:45.368195 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:45.368165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:45.455516 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:45.455489    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:46.060383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:46.060356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:48.010895 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:48.010839    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:49.574071 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:49.574028    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:51.562907 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:51.562837    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:51.668335 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:51.668275    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:51.766619 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:51.766470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:51.950173 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:51.950151    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:53.377818 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:53.377777    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:04:54.257346 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:54.257314    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:54.569828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:54.569654    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:54.930210 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:54.930086    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:55.397090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:55.397054    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:04:55.397874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:55.397825    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:04:55.616995 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:55.616889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:58.095311 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:58.095277    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:58.195407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:58.195359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:58.488040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:58.488005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:58.594570 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:58.594538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:59.669747 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:59.669713    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:59.762060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:59.762011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:04:59.871599 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:04:59.871563    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:00.308048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.307972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:00.408659 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.408596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:00.484913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.484880    2575 generic.go:356] "Generic (PLEG): container finished" podID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerID="96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423" exitCode=0
Sep 06 01:05:00.485076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.484945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" event={"ID":"8f2a85ab-8391-4f1f-8dfa-2101cc328435","Type":"ContainerDied","Data":"96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"}
Sep 06 01:05:00.485411 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.485393    2575 scope.go:117] "RemoveContainer" containerID="96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"
Sep 06 01:05:00.486647 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.486603    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:00.486746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.486701    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:00.486796 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.486753    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:00.486986 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.486969    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerID="b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4" exitCode=0
Sep 06 01:05:00.487037 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.487024    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" event={"ID":"d7a8e0eb-fdf5-4044-8f68-61f1f0791496","Type":"ContainerDied","Data":"b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"}
Sep 06 01:05:00.487358 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.487346    2575 scope.go:117] "RemoveContainer" containerID="b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"
Sep 06 01:05:00.488300 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.488273    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:00.488397 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.488333    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:00.488397 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:00.488364    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:01.499319 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:01.499284    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" event={"ID":"d7a8e0eb-fdf5-4044-8f68-61f1f0791496","Type":"ContainerStarted","Data":"e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7"}
Sep 06 01:05:01.504189 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:01.504150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" event={"ID":"8f2a85ab-8391-4f1f-8dfa-2101cc328435","Type":"ContainerStarted","Data":"c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e"}
Sep 06 01:05:01.635969 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:01.635933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:01.884921 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:01.884808    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:05:01.885299 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:01.885271    2575 patch_prober.go:28] interesting pod/e2e-autores-639343-openshift-default-684844c85d-b8f2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 06 01:05:01.885530 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:01.885502    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 06 01:05:02.295923 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:02.295891    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:05:02.296297 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:02.296209    2575 patch_prober.go:28] interesting pod/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 01:05:02.296445 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:02.296422    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 01:05:02.885666 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:02.885627    2575 patch_prober.go:28] interesting pod/e2e-autores-639343-openshift-default-684844c85d-b8f2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 06 01:05:02.886441 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:02.886410    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 06 01:05:03.296469 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:03.296429    2575 patch_prober.go:28] interesting pod/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 01:05:03.296773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:03.296744    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 01:05:03.522602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:03.522578    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:03.884999 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:03.884955    2575 patch_prober.go:28] interesting pod/e2e-autores-639343-openshift-default-684844c85d-b8f2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 06 01:05:03.885219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:03.885022    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 06 01:05:04.296134 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.296087    2575 patch_prober.go:28] interesting pod/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 01:05:04.296613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.296157    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 01:05:04.419331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.419299    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"]
Sep 06 01:05:04.422013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.421996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:04.424535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.424511    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:05:04.424725 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.424511    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:05:04.431555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.431531    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"]
Sep 06 01:05:04.515954 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.515913    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/600b3d21-2baa-4917-9944-2feb9d3560c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r\" (UID: \"600b3d21-2baa-4917-9944-2feb9d3560c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:04.617056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.616961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/600b3d21-2baa-4917-9944-2feb9d3560c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r\" (UID: \"600b3d21-2baa-4917-9944-2feb9d3560c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:04.617611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.617588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/600b3d21-2baa-4917-9944-2feb9d3560c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r\" (UID: \"600b3d21-2baa-4917-9944-2feb9d3560c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:04.733111 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.733075    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:04.886941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.886284    2575 patch_prober.go:28] interesting pod/e2e-autores-639343-openshift-default-684844c85d-b8f2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 06 01:05:04.886941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:04.886359    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 06 01:05:05.020139 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.020108    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:05.051894 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.051830    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"]
Sep 06 01:05:05.063037 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:05:05.062998    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600b3d21_2baa_4917_9944_2feb9d3560c3.slice/crio-710559aaa7460c98b71e546bd80adfd42700e19a167c599d206b003d2f38d398 WatchSource:0}: Error finding container 710559aaa7460c98b71e546bd80adfd42700e19a167c599d206b003d2f38d398: Status 404 returned error can't find the container with id 710559aaa7460c98b71e546bd80adfd42700e19a167c599d206b003d2f38d398
Sep 06 01:05:05.296373 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.296324    2575 patch_prober.go:28] interesting pod/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 01:05:05.296901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.296400    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 01:05:05.531026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.530994    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r" event={"ID":"600b3d21-2baa-4917-9944-2feb9d3560c3","Type":"ContainerStarted","Data":"710559aaa7460c98b71e546bd80adfd42700e19a167c599d206b003d2f38d398"}
Sep 06 01:05:05.799881 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.794976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:05.885645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.885207    2575 patch_prober.go:28] interesting pod/e2e-autores-639343-openshift-default-684844c85d-b8f2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 06 01:05:05.885645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:05.885268    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 06 01:05:06.296013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:06.295969    2575 patch_prober.go:28] interesting pod/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 06 01:05:06.296226 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:06.296038    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 06 01:05:07.157877 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:07.156731    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:05:07.157877 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:07.157111    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:05:07.158389 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:07.158146    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:05:07.371555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:07.371448    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:05:07.372219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:07.372155    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:05:07.374142 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:07.374103    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:05:08.551583 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:08.551543    2575 generic.go:356] "Generic (PLEG): container finished" podID="600b3d21-2baa-4917-9944-2feb9d3560c3" containerID="3276d286d4d22d4820a0442a506629e7d13e89a137f35032235bd3333b710009" exitCode=0
Sep 06 01:05:08.552017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:08.551682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r" event={"ID":"600b3d21-2baa-4917-9944-2feb9d3560c3","Type":"ContainerDied","Data":"3276d286d4d22d4820a0442a506629e7d13e89a137f35032235bd3333b710009"}
Sep 06 01:05:09.579668 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:09.579634    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:09.689150 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:09.689128    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:09.767099 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:09.767071    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/600b3d21-2baa-4917-9944-2feb9d3560c3-maas-api-service-ca\") pod \"600b3d21-2baa-4917-9944-2feb9d3560c3\" (UID: \"600b3d21-2baa-4917-9944-2feb9d3560c3\") "
Sep 06 01:05:09.767419 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:09.767395    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600b3d21-2baa-4917-9944-2feb9d3560c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "600b3d21-2baa-4917-9944-2feb9d3560c3" (UID: "600b3d21-2baa-4917-9944-2feb9d3560c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:05:09.868312 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:09.868234    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/600b3d21-2baa-4917-9944-2feb9d3560c3-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:10.565611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:10.565519    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r" event={"ID":"600b3d21-2baa-4917-9944-2feb9d3560c3","Type":"ContainerDied","Data":"710559aaa7460c98b71e546bd80adfd42700e19a167c599d206b003d2f38d398"}
Sep 06 01:05:10.565764 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:10.565530    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"
Sep 06 01:05:10.565764 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:10.565612    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="710559aaa7460c98b71e546bd80adfd42700e19a167c599d206b003d2f38d398"
Sep 06 01:05:20.345967 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:20.345926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:21.241180 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:21.241126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:21.925694 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:21.925657    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:21.975913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:21.975876    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"]
Sep 06 01:05:21.976245 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:21.976212    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2" podUID="5435d748-9689-4c45-a6ba-791f3bdda286" containerName="payload-pre-processing" containerID="cri-o://a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b" gracePeriod=30
Sep 06 01:05:22.029029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.029004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:22.125086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.125054    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:22.201378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.201292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:22.204183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.204158    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"]
Sep 06 01:05:22.204355 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.204199    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f8b122-mwp5r"]
Sep 06 01:05:22.242546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.242524    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:05:22.265604 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.265576    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:22.385716 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.385691    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd9qz\" (UniqueName: \"kubernetes.io/projected/5435d748-9689-4c45-a6ba-791f3bdda286-kube-api-access-bd9qz\") pod \"5435d748-9689-4c45-a6ba-791f3bdda286\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") "
Sep 06 01:05:22.385883 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.385742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5435d748-9689-4c45-a6ba-791f3bdda286-plugins-config-volume\") pod \"5435d748-9689-4c45-a6ba-791f3bdda286\" (UID: \"5435d748-9689-4c45-a6ba-791f3bdda286\") "
Sep 06 01:05:22.386183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.386162    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5435d748-9689-4c45-a6ba-791f3bdda286-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5435d748-9689-4c45-a6ba-791f3bdda286" (UID: "5435d748-9689-4c45-a6ba-791f3bdda286"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:05:22.388600 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.388306    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5435d748-9689-4c45-a6ba-791f3bdda286-kube-api-access-bd9qz" (OuterVolumeSpecName: "kube-api-access-bd9qz") pod "5435d748-9689-4c45-a6ba-791f3bdda286" (UID: "5435d748-9689-4c45-a6ba-791f3bdda286"). InnerVolumeSpecName "kube-api-access-bd9qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:05:22.417699 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.417660    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"]
Sep 06 01:05:22.417934 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.417912    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy" containerID="cri-o://c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e" gracePeriod=30
Sep 06 01:05:22.440326 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.440298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:22.486516 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.486454    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5435d748-9689-4c45-a6ba-791f3bdda286-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:22.486516 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.486478    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd9qz\" (UniqueName: \"kubernetes.io/projected/5435d748-9689-4c45-a6ba-791f3bdda286-kube-api-access-bd9qz\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:22.634424 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.634389    2575 generic.go:356] "Generic (PLEG): container finished" podID="5435d748-9689-4c45-a6ba-791f3bdda286" containerID="a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b" exitCode=0
Sep 06 01:05:22.634613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.634431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2" event={"ID":"5435d748-9689-4c45-a6ba-791f3bdda286","Type":"ContainerDied","Data":"a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b"}
Sep 06 01:05:22.634613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.634449    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"
Sep 06 01:05:22.634613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.634482    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2" event={"ID":"5435d748-9689-4c45-a6ba-791f3bdda286","Type":"ContainerDied","Data":"f12d8a648aed122f4aacecf1b0f78fc506b7608209216279ef55645e17798645"}
Sep 06 01:05:22.634613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.634507    2575 scope.go:117] "RemoveContainer" containerID="a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b"
Sep 06 01:05:22.643802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.643785    2575 scope.go:117] "RemoveContainer" containerID="a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b"
Sep 06 01:05:22.644151 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:05:22.644131    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b\": container with ID starting with a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b not found: ID does not exist" containerID="a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b"
Sep 06 01:05:22.644206 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.644160    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b"} err="failed to get container status \"a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b\": rpc error: code = NotFound desc = could not find container \"a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b\": container with ID starting with a1b553f6ae2d19c9d33090213e59a167eafcef44ce917576c26b01c98492b31b not found: ID does not exist"
Sep 06 01:05:22.655606 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.655581    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"]
Sep 06 01:05:22.659268 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:22.659248    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d3f8b122-8c59f54db-bv8h2"]
Sep 06 01:05:23.353333 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:23.353230    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:23.446138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:23.446104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:23.701354 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:23.701329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:23.800444 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:23.800406    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:24.167272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167186    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"]
Sep 06 01:05:24.167618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167603    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5435d748-9689-4c45-a6ba-791f3bdda286" containerName="payload-pre-processing"
Sep 06 01:05:24.167662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167621    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5435d748-9689-4c45-a6ba-791f3bdda286" containerName="payload-pre-processing"
Sep 06 01:05:24.167662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167630    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600b3d21-2baa-4917-9944-2feb9d3560c3" containerName="revoke-keys"
Sep 06 01:05:24.167662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167636    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="600b3d21-2baa-4917-9944-2feb9d3560c3" containerName="revoke-keys"
Sep 06 01:05:24.167754 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167695    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="600b3d21-2baa-4917-9944-2feb9d3560c3" containerName="revoke-keys"
Sep 06 01:05:24.167754 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.167705    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5435d748-9689-4c45-a6ba-791f3bdda286" containerName="payload-pre-processing"
Sep 06 01:05:24.170513 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.170493    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.172888 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.172871    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2460bb62-gw-openshift-default-dockercfg-rzzj8\""
Sep 06 01:05:24.191229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.191202    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"]
Sep 06 01:05:24.200367 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmkh9\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-kube-api-access-jmkh9\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200404    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-token\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-envoy\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-data\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200520    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-podinfo\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-credential-socket\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-socket\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200652    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-certs\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.200695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.200680    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istiod-ca-cert\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.202547 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.202522    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5435d748-9689-4c45-a6ba-791f3bdda286" path="/var/lib/kubelet/pods/5435d748-9689-4c45-a6ba-791f3bdda286/volumes"
Sep 06 01:05:24.202892 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.202843    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600b3d21-2baa-4917-9944-2feb9d3560c3" path="/var/lib/kubelet/pods/600b3d21-2baa-4917-9944-2feb9d3560c3/volumes"
Sep 06 01:05:24.223913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.223884    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:24.302024 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.301991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-credential-socket\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302209 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302032    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-socket\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302209 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302051    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-certs\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302209 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istiod-ca-cert\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302209 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302123    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmkh9\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-kube-api-access-jmkh9\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302209 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-token\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-credential-socket\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-envoy\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-data\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302631 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-podinfo\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302631 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302502    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-socket\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302631 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-certs\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302764 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302739    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-data\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.302951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.302934    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istiod-ca-cert\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.304904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.304881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-envoy\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.305142 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.305123    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-podinfo\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.310562 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.310537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-token\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.310643 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.310583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmkh9\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-kube-api-access-jmkh9\") pod \"e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") " pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.480845 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.480811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:24.556420 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.556374    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:24.762108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.762083    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"]
Sep 06 01:05:24.763744 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:05:24.763718    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda307ff95_e31c_48f7_bb4c_60e06fb5321d.slice/crio-f71a048875d0d1a6724301c92685e4b43bbb53e8c32edadd78a22178cd8b2b2d WatchSource:0}: Error finding container f71a048875d0d1a6724301c92685e4b43bbb53e8c32edadd78a22178cd8b2b2d: Status 404 returned error can't find the container with id f71a048875d0d1a6724301c92685e4b43bbb53e8c32edadd78a22178cd8b2b2d
Sep 06 01:05:24.766274 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.766243    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:24.766374 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.766305    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:24.766374 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:24.766330    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:25.653239 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:25.653203    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" event={"ID":"a307ff95-e31c-48f7-bb4c-60e06fb5321d","Type":"ContainerStarted","Data":"e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"}
Sep 06 01:05:25.653629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:25.653252    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" event={"ID":"a307ff95-e31c-48f7-bb4c-60e06fb5321d","Type":"ContainerStarted","Data":"f71a048875d0d1a6724301c92685e4b43bbb53e8c32edadd78a22178cd8b2b2d"}
Sep 06 01:05:25.682549 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:25.682503    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podStartSLOduration=1.6824900980000002 podStartE2EDuration="1.682490098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:05:25.679242161 +0000 UTC m=+888.008600996" watchObservedRunningTime="2026-09-06 01:05:25.682490098 +0000 UTC m=+888.011848930"
Sep 06 01:05:26.482026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:26.481990    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:26.487592 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:26.487567    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:26.659155 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:26.659127    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:26.660015 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:26.659996    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:27.224105 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.224074    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:27.318077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.318047    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:27.476091 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.476019    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:27.577108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.577076    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:27.665602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.665579    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:05:27.667432 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.667408    2575 generic.go:356] "Generic (PLEG): container finished" podID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerID="c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e" exitCode=0
Sep 06 01:05:27.667533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.667454    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"
Sep 06 01:05:27.667533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.667489    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" event={"ID":"8f2a85ab-8391-4f1f-8dfa-2101cc328435","Type":"ContainerDied","Data":"c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e"}
Sep 06 01:05:27.667605 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.667539    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb" event={"ID":"8f2a85ab-8391-4f1f-8dfa-2101cc328435","Type":"ContainerDied","Data":"46c16e60a4d44fbc5ad35469bb1b88c4990185a99d5bfd25ffec9fd8bdee68b5"}
Sep 06 01:05:27.667605 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.667568    2575 scope.go:117] "RemoveContainer" containerID="c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e"
Sep 06 01:05:27.677556 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.677538    2575 scope.go:117] "RemoveContainer" containerID="96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"
Sep 06 01:05:27.685887 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.685845    2575 scope.go:117] "RemoveContainer" containerID="c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e"
Sep 06 01:05:27.686165 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:05:27.686148    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e\": container with ID starting with c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e not found: ID does not exist" containerID="c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e"
Sep 06 01:05:27.686213 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.686176    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e"} err="failed to get container status \"c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e\": rpc error: code = NotFound desc = could not find container \"c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e\": container with ID starting with c985640f9836bc2e71aad4096b06c4e6a383de0d0f2341a6cf993cd538493b5e not found: ID does not exist"
Sep 06 01:05:27.686213 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.686193    2575 scope.go:117] "RemoveContainer" containerID="96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"
Sep 06 01:05:27.686477 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:05:27.686460    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423\": container with ID starting with 96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423 not found: ID does not exist" containerID="96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"
Sep 06 01:05:27.686521 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.686485    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423"} err="failed to get container status \"96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423\": rpc error: code = NotFound desc = could not find container \"96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423\": container with ID starting with 96b9a4ba7545a410183e353d7f5d80e0d91830834c50a7de204e10028c9f0423 not found: ID does not exist"
Sep 06 01:05:27.735220 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735144    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-token\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735220 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735179    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-certs\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735220 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-data\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735243    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-socket\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735292    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istiod-ca-cert\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735328    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-envoy\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735353    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-credential-socket\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735386    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkcpm\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-kube-api-access-gkcpm\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735433    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-podinfo\") pod \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\" (UID: \"8f2a85ab-8391-4f1f-8dfa-2101cc328435\") "
Sep 06 01:05:27.735756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735543    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:05:27.735756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735552    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:05:27.735756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735568    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:05:27.735934 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735885    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:05:27.735934 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.735917    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:05:27.736252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.736181    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.736252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.736207    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.736252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.736223    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.737775 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.737753    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:05:27.737775 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.737764    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-kube-api-access-gkcpm" (OuterVolumeSpecName: "kube-api-access-gkcpm") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "kube-api-access-gkcpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:05:27.738013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.737994    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:05:27.738059 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.738017    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f2a85ab-8391-4f1f-8dfa-2101cc328435" (UID: "8f2a85ab-8391-4f1f-8dfa-2101cc328435"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:05:27.837425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.837392    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.837425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.837423    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.837425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.837432    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.837653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.837440    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f2a85ab-8391-4f1f-8dfa-2101cc328435-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.837653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.837449    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkcpm\" (UniqueName: \"kubernetes.io/projected/8f2a85ab-8391-4f1f-8dfa-2101cc328435-kube-api-access-gkcpm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.837653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.837458    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f2a85ab-8391-4f1f-8dfa-2101cc328435-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:27.947329 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.947295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:27.997225 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:27.997154    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"]
Sep 06 01:05:28.003792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.003762    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d3f8b122-openshift-default-7fb97b7488-r8jwb"]
Sep 06 01:05:28.045973 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.045917    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:28.146114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.146084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:28.165081 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165052    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"]
Sep 06 01:05:28.165480 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165467    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy"
Sep 06 01:05:28.165517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165483    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy"
Sep 06 01:05:28.165517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165499    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy"
Sep 06 01:05:28.165517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165506    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy"
Sep 06 01:05:28.165617 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165567    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy"
Sep 06 01:05:28.165617 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.165575    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" containerName="istio-proxy"
Sep 06 01:05:28.168357 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.168341    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.170943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.170914    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-2460bb62\""
Sep 06 01:05:28.171039 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.170995    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-2460bb62-dockercfg-qdznq\""
Sep 06 01:05:28.181529 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.181510    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"]
Sep 06 01:05:28.204632 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.204600    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f2a85ab-8391-4f1f-8dfa-2101cc328435" path="/var/lib/kubelet/pods/8f2a85ab-8391-4f1f-8dfa-2101cc328435/volumes"
Sep 06 01:05:28.236261 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.236227    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"]
Sep 06 01:05:28.238636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.238620    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.240422 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.240389    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsc65\" (UniqueName: \"kubernetes.io/projected/89704443-6b89-4d0b-8062-8788a6d5f556-kube-api-access-hsc65\") pod \"payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.240593 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.240573    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89704443-6b89-4d0b-8062-8788a6d5f556-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.250373 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.250313    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"]
Sep 06 01:05:28.341930 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.341848    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") " pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.342118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.341943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gr5b\" (UniqueName: \"kubernetes.io/projected/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-kube-api-access-4gr5b\") pod \"payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") " pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.342118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.342001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89704443-6b89-4d0b-8062-8788a6d5f556-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.342118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.342026    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsc65\" (UniqueName: \"kubernetes.io/projected/89704443-6b89-4d0b-8062-8788a6d5f556-kube-api-access-hsc65\") pod \"payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.342644 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.342622    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89704443-6b89-4d0b-8062-8788a6d5f556-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.352611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.352586    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsc65\" (UniqueName: \"kubernetes.io/projected/89704443-6b89-4d0b-8062-8788a6d5f556-kube-api-access-hsc65\") pod \"payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.443237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.443204    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") " pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.443237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.443236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gr5b\" (UniqueName: \"kubernetes.io/projected/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-kube-api-access-4gr5b\") pod \"payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") " pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.443913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.443894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") " pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.454346 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.454324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gr5b\" (UniqueName: \"kubernetes.io/projected/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-kube-api-access-4gr5b\") pod \"payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") " pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.479061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.479030    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:28.489688 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.489661    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:28.549669 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.549538    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:28.755447 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.755424    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"]
Sep 06 01:05:28.756284 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:05:28.756258    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89704443_6b89_4d0b_8062_8788a6d5f556.slice/crio-ee9c5bf74f06f09f2d2fb96fc26255d38b6fad18032196c82a2e33edd1adffe0 WatchSource:0}: Error finding container ee9c5bf74f06f09f2d2fb96fc26255d38b6fad18032196c82a2e33edd1adffe0: Status 404 returned error can't find the container with id ee9c5bf74f06f09f2d2fb96fc26255d38b6fad18032196c82a2e33edd1adffe0
Sep 06 01:05:28.836135 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:28.836100    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"]
Sep 06 01:05:29.681816 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.681780    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24" event={"ID":"89704443-6b89-4d0b-8062-8788a6d5f556","Type":"ContainerStarted","Data":"ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6"}
Sep 06 01:05:29.682020 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.681834    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24" event={"ID":"89704443-6b89-4d0b-8062-8788a6d5f556","Type":"ContainerStarted","Data":"ee9c5bf74f06f09f2d2fb96fc26255d38b6fad18032196c82a2e33edd1adffe0"}
Sep 06 01:05:29.682020 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.681886    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:29.683581 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.683554    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz" event={"ID":"0dde0f70-ce9b-4e1b-990a-c16050fc22ec","Type":"ContainerStarted","Data":"85ed3bc12d7716d95fa343b2d04d781cc75d1fb2e688625f21ff021f3050db7d"}
Sep 06 01:05:29.683669 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.683608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz" event={"ID":"0dde0f70-ce9b-4e1b-990a-c16050fc22ec","Type":"ContainerStarted","Data":"ce80d775b1eaad06bfb8f01db62e3ffce1384862183f07a9cfe0aff2fe26eaf3"}
Sep 06 01:05:29.683669 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.683635    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:29.704421 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.704371    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24" podStartSLOduration=1.704352139 podStartE2EDuration="1.704352139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:05:29.702412284 +0000 UTC m=+892.031771139" watchObservedRunningTime="2026-09-06 01:05:29.704352139 +0000 UTC m=+892.033710979"
Sep 06 01:05:29.724374 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:29.724333    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz" podStartSLOduration=1.724316849 podStartE2EDuration="1.724316849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:05:29.723094505 +0000 UTC m=+892.052453366" watchObservedRunningTime="2026-09-06 01:05:29.724316849 +0000 UTC m=+892.053675686"
Sep 06 01:05:38.203724 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:38.203689    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:05:38.204408 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:38.204387    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:05:40.692521 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:40.692469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:40.693049 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:40.692565    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:41.661334 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.661300    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 06 01:05:41.661485 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.661363    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 06 01:05:41.757276 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.757210    2575 generic.go:356] "Generic (PLEG): container finished" podID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerID="e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d" exitCode=0
Sep 06 01:05:41.757602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.757285    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" event={"ID":"a307ff95-e31c-48f7-bb4c-60e06fb5321d","Type":"ContainerDied","Data":"e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"}
Sep 06 01:05:41.757670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.757654    2575 scope.go:117] "RemoveContainer" containerID="e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"
Sep 06 01:05:41.758727 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.758703    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:41.758778 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.758767    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:41.758833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:41.758822    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:05:42.773586 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:42.773540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" event={"ID":"a307ff95-e31c-48f7-bb4c-60e06fb5321d","Type":"ContainerStarted","Data":"d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31"}
Sep 06 01:05:43.481507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:43.481465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:43.481722 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:43.481696    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 06 01:05:43.481775 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:43.481750    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 06 01:05:43.716663 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:43.716623    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:44.481869 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:44.481820    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 06 01:05:44.482280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:44.481902    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 06 01:05:45.481908 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:45.481838    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 06 01:05:45.482292 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:45.481952    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 06 01:05:46.482183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:46.482139    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 06 01:05:46.482573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:46.482209    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 06 01:05:47.481652 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:47.481606    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 06 01:05:47.481833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:47.481689    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 06 01:05:48.485544 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:48.485510    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:48.486032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:48.485768    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:48.486471 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:48.486449    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:05:50.653251 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.653220    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"]
Sep 06 01:05:50.656044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.656021    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.660552 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.660532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-639343\"/\"autores-model-639343-kserve-self-signed-certs\""
Sep 06 01:05:50.660679 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.660536    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-639343\"/\"default-dockercfg-7jvmb\""
Sep 06 01:05:50.660679 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.660537    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-639343\"/\"openshift-service-ca.crt\""
Sep 06 01:05:50.660679 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.660565    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-639343\"/\"kube-root-ca.crt\""
Sep 06 01:05:50.666639 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.666616    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"]
Sep 06 01:05:50.676464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.676441    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:50.767324 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.767298    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-dshm\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.767438 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.767329    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-tmp-dir\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.767438 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.767411    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4vt8\" (UniqueName: \"kubernetes.io/projected/0a2e7306-6296-4f26-890f-e6556f188aad-kube-api-access-l4vt8\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.767514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.767437    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2e7306-6296-4f26-890f-e6556f188aad-tls-certs\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.767514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.767472    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-home\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.767514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.767487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-model-cache\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868324 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4vt8\" (UniqueName: \"kubernetes.io/projected/0a2e7306-6296-4f26-890f-e6556f188aad-kube-api-access-l4vt8\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2e7306-6296-4f26-890f-e6556f188aad-tls-certs\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-home\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-model-cache\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-dshm\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868457    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-tmp-dir\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868809 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868787    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-home\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868834    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-tmp-dir\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.868903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.868875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-model-cache\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.870974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.870952    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-dshm\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.871220 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.871202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2e7306-6296-4f26-890f-e6556f188aad-tls-certs\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.876438 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.876418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4vt8\" (UniqueName: \"kubernetes.io/projected/0a2e7306-6296-4f26-890f-e6556f188aad-kube-api-access-l4vt8\") pod \"autores-model-639343-kserve-c95947bb7-6hz44\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") " pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:50.965759 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:50.965723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:05:51.235889 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:51.235839    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"]
Sep 06 01:05:51.236744 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:05:51.236721    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a2e7306_6296_4f26_890f_e6556f188aad.slice/crio-0938ad0d6c5f4d3bae55c3f74767d0e57a417f079be7e97b8da56cd49a561ef7 WatchSource:0}: Error finding container 0938ad0d6c5f4d3bae55c3f74767d0e57a417f079be7e97b8da56cd49a561ef7: Status 404 returned error can't find the container with id 0938ad0d6c5f4d3bae55c3f74767d0e57a417f079be7e97b8da56cd49a561ef7
Sep 06 01:05:51.827591 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:51.827555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44" event={"ID":"0a2e7306-6296-4f26-890f-e6556f188aad","Type":"ContainerStarted","Data":"89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df"}
Sep 06 01:05:51.828013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:51.827600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44" event={"ID":"0a2e7306-6296-4f26-890f-e6556f188aad","Type":"ContainerStarted","Data":"0938ad0d6c5f4d3bae55c3f74767d0e57a417f079be7e97b8da56cd49a561ef7"}
Sep 06 01:05:51.843947 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:51.843906    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44" podStartSLOduration=1.843891278 podStartE2EDuration="1.843891278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:05:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:05:51.843205467 +0000 UTC m=+914.172564327" watchObservedRunningTime="2026-09-06 01:05:51.843891278 +0000 UTC m=+914.173250149"
Sep 06 01:05:52.785712 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:52.785674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:58.713382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.713347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:58.744816 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.744785    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"]
Sep 06 01:05:58.745027 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.745002    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz" podUID="0dde0f70-ce9b-4e1b-990a-c16050fc22ec" containerName="payload-processing" containerID="cri-o://85ed3bc12d7716d95fa343b2d04d781cc75d1fb2e688625f21ff021f3050db7d" gracePeriod=30
Sep 06 01:05:58.755095 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.755067    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"]
Sep 06 01:05:58.755367 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.755289    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24" podUID="89704443-6b89-4d0b-8062-8788a6d5f556" containerName="payload-pre-processing" containerID="cri-o://ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6" gracePeriod=30
Sep 06 01:05:58.787485 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.787458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:58.857811 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.857761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:58.868571 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.868517    2575 generic.go:356] "Generic (PLEG): container finished" podID="0dde0f70-ce9b-4e1b-990a-c16050fc22ec" containerID="85ed3bc12d7716d95fa343b2d04d781cc75d1fb2e688625f21ff021f3050db7d" exitCode=0
Sep 06 01:05:58.868666 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.868584    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz" event={"ID":"0dde0f70-ce9b-4e1b-990a-c16050fc22ec","Type":"ContainerDied","Data":"85ed3bc12d7716d95fa343b2d04d781cc75d1fb2e688625f21ff021f3050db7d"}
Sep 06 01:05:58.928388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:58.928348    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:59.027010 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.026983    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:59.056146 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.056119    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:59.071883 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.069218    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:59.107494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.107453    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"]
Sep 06 01:05:59.107878 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.107795    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" containerID="cri-o://d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31" gracePeriod=30
Sep 06 01:05:59.134197 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.134170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:59.142228 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.142205    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsc65\" (UniqueName: \"kubernetes.io/projected/89704443-6b89-4d0b-8062-8788a6d5f556-kube-api-access-hsc65\") pod \"89704443-6b89-4d0b-8062-8788a6d5f556\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") "
Sep 06 01:05:59.142359 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.142248    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gr5b\" (UniqueName: \"kubernetes.io/projected/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-kube-api-access-4gr5b\") pod \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") "
Sep 06 01:05:59.142359 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.142301    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89704443-6b89-4d0b-8062-8788a6d5f556-plugins-config-volume\") pod \"89704443-6b89-4d0b-8062-8788a6d5f556\" (UID: \"89704443-6b89-4d0b-8062-8788a6d5f556\") "
Sep 06 01:05:59.142471 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.142365    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-plugins-config-volume\") pod \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\" (UID: \"0dde0f70-ce9b-4e1b-990a-c16050fc22ec\") "
Sep 06 01:05:59.142780 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.142746    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89704443-6b89-4d0b-8062-8788a6d5f556-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89704443-6b89-4d0b-8062-8788a6d5f556" (UID: "89704443-6b89-4d0b-8062-8788a6d5f556"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:05:59.143012 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.142983    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0dde0f70-ce9b-4e1b-990a-c16050fc22ec" (UID: "0dde0f70-ce9b-4e1b-990a-c16050fc22ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:05:59.144717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.144686    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-kube-api-access-4gr5b" (OuterVolumeSpecName: "kube-api-access-4gr5b") pod "0dde0f70-ce9b-4e1b-990a-c16050fc22ec" (UID: "0dde0f70-ce9b-4e1b-990a-c16050fc22ec"). InnerVolumeSpecName "kube-api-access-4gr5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:05:59.145035 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.145018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89704443-6b89-4d0b-8062-8788a6d5f556-kube-api-access-hsc65" (OuterVolumeSpecName: "kube-api-access-hsc65") pod "89704443-6b89-4d0b-8062-8788a6d5f556" (UID: "89704443-6b89-4d0b-8062-8788a6d5f556"). InnerVolumeSpecName "kube-api-access-hsc65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:05:59.244204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.244106    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89704443-6b89-4d0b-8062-8788a6d5f556-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:59.244204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.244139    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:59.244204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.244148    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsc65\" (UniqueName: \"kubernetes.io/projected/89704443-6b89-4d0b-8062-8788a6d5f556-kube-api-access-hsc65\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:59.244204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.244156    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gr5b\" (UniqueName: \"kubernetes.io/projected/0dde0f70-ce9b-4e1b-990a-c16050fc22ec-kube-api-access-4gr5b\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:05:59.551238 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.551132    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:59.828150 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828059    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"]
Sep 06 01:05:59.828563 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828546    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dde0f70-ce9b-4e1b-990a-c16050fc22ec" containerName="payload-processing"
Sep 06 01:05:59.828610 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828566    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dde0f70-ce9b-4e1b-990a-c16050fc22ec" containerName="payload-processing"
Sep 06 01:05:59.828610 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828582    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89704443-6b89-4d0b-8062-8788a6d5f556" containerName="payload-pre-processing"
Sep 06 01:05:59.828610 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828588    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="89704443-6b89-4d0b-8062-8788a6d5f556" containerName="payload-pre-processing"
Sep 06 01:05:59.828709 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828650    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dde0f70-ce9b-4e1b-990a-c16050fc22ec" containerName="payload-processing"
Sep 06 01:05:59.828709 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.828660    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="89704443-6b89-4d0b-8062-8788a6d5f556" containerName="payload-pre-processing"
Sep 06 01:05:59.830931 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.830911    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.833893 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.833836    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6cee6e60-openshift-default-dockercfg-tssfv\""
Sep 06 01:05:59.842235 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.842215    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"]
Sep 06 01:05:59.853903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.853841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:05:59.876914 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.876876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz" event={"ID":"0dde0f70-ce9b-4e1b-990a-c16050fc22ec","Type":"ContainerDied","Data":"ce80d775b1eaad06bfb8f01db62e3ffce1384862183f07a9cfe0aff2fe26eaf3"}
Sep 06 01:05:59.877031 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.876931    2575 scope.go:117] "RemoveContainer" containerID="85ed3bc12d7716d95fa343b2d04d781cc75d1fb2e688625f21ff021f3050db7d"
Sep 06 01:05:59.877031 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.876948    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"
Sep 06 01:05:59.878512 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.878467    2575 generic.go:356] "Generic (PLEG): container finished" podID="89704443-6b89-4d0b-8062-8788a6d5f556" containerID="ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6" exitCode=0
Sep 06 01:05:59.878512 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.878499    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24" event={"ID":"89704443-6b89-4d0b-8062-8788a6d5f556","Type":"ContainerDied","Data":"ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6"}
Sep 06 01:05:59.878698 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.878520    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"
Sep 06 01:05:59.878698 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.878547    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24" event={"ID":"89704443-6b89-4d0b-8062-8788a6d5f556","Type":"ContainerDied","Data":"ee9c5bf74f06f09f2d2fb96fc26255d38b6fad18032196c82a2e33edd1adffe0"}
Sep 06 01:05:59.888499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.888474    2575 scope.go:117] "RemoveContainer" containerID="ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6"
Sep 06 01:05:59.897360 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.897338    2575 scope.go:117] "RemoveContainer" containerID="ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6"
Sep 06 01:05:59.897673 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:05:59.897653    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6\": container with ID starting with ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6 not found: ID does not exist" containerID="ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6"
Sep 06 01:05:59.897746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.897683    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6"} err="failed to get container status \"ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6\": rpc error: code = NotFound desc = could not find container \"ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6\": container with ID starting with ea2eb07192fabb033275ea4775e7f7df0fb341960ff660a351e13bf099a5b2c6 not found: ID does not exist"
Sep 06 01:05:59.906496 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.906475    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"]
Sep 06 01:05:59.910681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.910643    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2460bb62-657b48f474-4sspz"]
Sep 06 01:05:59.922235 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.922204    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"]
Sep 06 01:05:59.932241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.932214    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2460bb62-6c775c86b6-f4z24"]
Sep 06 01:05:59.951375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-certs\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d572b-9b25-4751-b01b-f2faee69df80-istio-podinfo\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-envoy\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-socket\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951542    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-data\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-istio-token\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7msp\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-kube-api-access-k7msp\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951663    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d572b-9b25-4751-b01b-f2faee69df80-istiod-ca-cert\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:05:59.951736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:05:59.951682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-credential-socket\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.006706 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.006674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:00.053186 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-istio-token\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053186 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053189    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7msp\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-kube-api-access-k7msp\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d572b-9b25-4751-b01b-f2faee69df80-istiod-ca-cert\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053245    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-credential-socket\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053302    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-certs\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d572b-9b25-4751-b01b-f2faee69df80-istio-podinfo\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-envoy\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-socket\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-data\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053795 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-credential-socket\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.053907 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053886    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-data\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.054003 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.053901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-socket\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.054089 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.054064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d572b-9b25-4751-b01b-f2faee69df80-istiod-ca-cert\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.054142 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.054125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-certs\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.056534 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.056510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-envoy\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.056714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.056692    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d572b-9b25-4751-b01b-f2faee69df80-istio-podinfo\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.061361 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.061338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-istio-token\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.061613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.061594    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7msp\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-kube-api-access-k7msp\") pod \"e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") " pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.142644 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.142565    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:00.205441 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.205382    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dde0f70-ce9b-4e1b-990a-c16050fc22ec" path="/var/lib/kubelet/pods/0dde0f70-ce9b-4e1b-990a-c16050fc22ec/volumes"
Sep 06 01:06:00.207361 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.205888    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89704443-6b89-4d0b-8062-8788a6d5f556" path="/var/lib/kubelet/pods/89704443-6b89-4d0b-8062-8788a6d5f556/volumes"
Sep 06 01:06:00.234204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.234181    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:00.425008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.424966    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"]
Sep 06 01:06:00.426159 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:00.426128    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434d572b_9b25_4751_b01b_f2faee69df80.slice/crio-48c08739bfb0e4c35c78b7fcc5059727589546b85029ac94ef5f4f1b42deb527 WatchSource:0}: Error finding container 48c08739bfb0e4c35c78b7fcc5059727589546b85029ac94ef5f4f1b42deb527: Status 404 returned error can't find the container with id 48c08739bfb0e4c35c78b7fcc5059727589546b85029ac94ef5f4f1b42deb527
Sep 06 01:06:00.428846 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.428808    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:00.428945 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.428916    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:00.428989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.428962    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:00.886184 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.886146    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" event={"ID":"434d572b-9b25-4751-b01b-f2faee69df80","Type":"ContainerStarted","Data":"853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"}
Sep 06 01:06:00.886647 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.886197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" event={"ID":"434d572b-9b25-4751-b01b-f2faee69df80","Type":"ContainerStarted","Data":"48c08739bfb0e4c35c78b7fcc5059727589546b85029ac94ef5f4f1b42deb527"}
Sep 06 01:06:00.906971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.906926    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podStartSLOduration=1.90691152 podStartE2EDuration="1.90691152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:00.906038987 +0000 UTC m=+923.235397839" watchObservedRunningTime="2026-09-06 01:06:00.90691152 +0000 UTC m=+923.236270356"
Sep 06 01:06:00.966545 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.966520    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:06:00.966773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.966755    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:06:00.979126 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:00.979105    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:06:01.142943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:01.142839    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:01.147996 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:01.147975    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:01.893095 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:01.893056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:01.894073 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:01.894055    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:01.904155 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:01.904134    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:06:02.259433 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:02.259391    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:02.448423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:02.448392    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:03.439239 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.439198    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:03.488126 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.488095    2575 patch_prober.go:28] interesting pod/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 01:06:03.488283 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.488164    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 01:06:03.507206 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.507177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:03.761447 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.761393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:03.897180 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.897142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:03.999996 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:03.999963    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:04.366661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.366639    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:06:04.497503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497470    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istiod-ca-cert\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.497503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497505    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-socket\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497536    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-envoy\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497562    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-token\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497594    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmkh9\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-kube-api-access-jmkh9\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497609    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-data\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497623    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-credential-socket\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497646    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-certs\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497723    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-podinfo\") pod \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\" (UID: \"a307ff95-e31c-48f7-bb4c-60e06fb5321d\") "
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497781    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.497992    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:04.498064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.498033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:04.498740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.498171    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.498740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.498190    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.498740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.498201    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.498740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.498198    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:04.498740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.498252    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:04.500011 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.499990    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:04.500011 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.499998    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-kube-api-access-jmkh9" (OuterVolumeSpecName: "kube-api-access-jmkh9") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "kube-api-access-jmkh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:04.500395 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.500376    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:04.500395 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.500383    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a307ff95-e31c-48f7-bb4c-60e06fb5321d" (UID: "a307ff95-e31c-48f7-bb4c-60e06fb5321d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:06:04.599257 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.599224    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.599257 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.599255    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.599427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.599267    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.599427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.599278    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmkh9\" (UniqueName: \"kubernetes.io/projected/a307ff95-e31c-48f7-bb4c-60e06fb5321d-kube-api-access-jmkh9\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.599427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.599291    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.599427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.599302    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a307ff95-e31c-48f7-bb4c-60e06fb5321d-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:04.911065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.910983    2575 generic.go:356] "Generic (PLEG): container finished" podID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerID="d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31" exitCode=0
Sep 06 01:06:04.911065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.911033    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"
Sep 06 01:06:04.911250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.911065    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" event={"ID":"a307ff95-e31c-48f7-bb4c-60e06fb5321d","Type":"ContainerDied","Data":"d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31"}
Sep 06 01:06:04.911250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.911123    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn" event={"ID":"a307ff95-e31c-48f7-bb4c-60e06fb5321d","Type":"ContainerDied","Data":"f71a048875d0d1a6724301c92685e4b43bbb53e8c32edadd78a22178cd8b2b2d"}
Sep 06 01:06:04.911250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.911153    2575 scope.go:117] "RemoveContainer" containerID="d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31"
Sep 06 01:06:04.921519 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.921502    2575 scope.go:117] "RemoveContainer" containerID="e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"
Sep 06 01:06:04.932596 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.932577    2575 scope.go:117] "RemoveContainer" containerID="d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31"
Sep 06 01:06:04.932970 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:04.932940    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31\": container with ID starting with d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31 not found: ID does not exist" containerID="d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31"
Sep 06 01:06:04.933076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.932980    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31"} err="failed to get container status \"d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31\": rpc error: code = NotFound desc = could not find container \"d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31\": container with ID starting with d511dd0f56be1e9d83e970094826e69ef4a60ec86d34509ece02989fd9700a31 not found: ID does not exist"
Sep 06 01:06:04.933076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.932999    2575 scope.go:117] "RemoveContainer" containerID="e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"
Sep 06 01:06:04.933265 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:04.933247    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d\": container with ID starting with e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d not found: ID does not exist" containerID="e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"
Sep 06 01:06:04.933310 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.933273    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d"} err="failed to get container status \"e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d\": rpc error: code = NotFound desc = could not find container \"e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d\": container with ID starting with e64dc8ef23d9f9f6c3b5c06fbdca00b5dbe4ef35815c8fd29fc3f10207a6397d not found: ID does not exist"
Sep 06 01:06:04.933836 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.933818    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"]
Sep 06 01:06:04.937726 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:04.937707    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2460bb62-gw-openshift-default-5845b44c68-l57zn"]
Sep 06 01:06:06.201777 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:06.201741    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" path="/var/lib/kubelet/pods/a307ff95-e31c-48f7-bb4c-60e06fb5321d/volumes"
Sep 06 01:06:07.650319 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650282    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"]
Sep 06 01:06:07.650691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650636    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy"
Sep 06 01:06:07.650691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650646    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy"
Sep 06 01:06:07.650691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650663    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy"
Sep 06 01:06:07.650691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650668    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy"
Sep 06 01:06:07.650812 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650725    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy"
Sep 06 01:06:07.650812 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.650734    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a307ff95-e31c-48f7-bb4c-60e06fb5321d" containerName="istio-proxy"
Sep 06 01:06:07.653647 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.653632    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.656316 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.656292    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-701b5b\"/\"openshift-service-ca.crt\""
Sep 06 01:06:07.657114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.657094    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-701b5b\"/\"kube-root-ca.crt\""
Sep 06 01:06:07.657200 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.657141    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-701b5b\"/\"test-model-701b5b-kserve-self-signed-certs\""
Sep 06 01:06:07.657200 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.657103    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-701b5b\"/\"default-dockercfg-98tpb\""
Sep 06 01:06:07.663426 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.663064    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"]
Sep 06 01:06:07.825646 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.825615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tls-certs\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.825797 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.825670    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-dshm\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.825797 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.825691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz9cv\" (UniqueName: \"kubernetes.io/projected/ca30e9ec-7602-4745-8fd9-1fba1e55129a-kube-api-access-cz9cv\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.825797 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.825767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tmp-dir\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.825952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.825809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-model-cache\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.825952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.825912    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-home\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.926874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.926782    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-home\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.926874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.926825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tls-certs\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.926876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-dshm\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.926901    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz9cv\" (UniqueName: \"kubernetes.io/projected/ca30e9ec-7602-4745-8fd9-1fba1e55129a-kube-api-access-cz9cv\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.926933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tmp-dir\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.926959    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-model-cache\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.927255    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-home\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.927324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-model-cache\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.927451 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.927372    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tmp-dir\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.929305 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.929283    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-dshm\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.929613 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.929593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tls-certs\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.937154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.937122    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz9cv\" (UniqueName: \"kubernetes.io/projected/ca30e9ec-7602-4745-8fd9-1fba1e55129a-kube-api-access-cz9cv\") pod \"test-model-701b5b-kserve-7858485947-8jh27\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") " pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:07.965909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:07.965888    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:08.055831 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:08.055797    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:08.240752 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:08.240711    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"]
Sep 06 01:06:08.242256 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:08.242229    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca30e9ec_7602_4745_8fd9_1fba1e55129a.slice/crio-46f2ad598be0b5b7bc4bf342baddbebe47b1b0b388a9dd7a0f1bd229fea5df73 WatchSource:0}: Error finding container 46f2ad598be0b5b7bc4bf342baddbebe47b1b0b388a9dd7a0f1bd229fea5df73: Status 404 returned error can't find the container with id 46f2ad598be0b5b7bc4bf342baddbebe47b1b0b388a9dd7a0f1bd229fea5df73
Sep 06 01:06:08.936807 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:08.936768    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27" event={"ID":"ca30e9ec-7602-4745-8fd9-1fba1e55129a","Type":"ContainerStarted","Data":"fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d"}
Sep 06 01:06:08.937269 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:08.936835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27" event={"ID":"ca30e9ec-7602-4745-8fd9-1fba1e55129a","Type":"ContainerStarted","Data":"46f2ad598be0b5b7bc4bf342baddbebe47b1b0b388a9dd7a0f1bd229fea5df73"}
Sep 06 01:06:08.954886 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:08.954816    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27" podStartSLOduration=1.954800955 podStartE2EDuration="1.954800955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:08.952789061 +0000 UTC m=+931.282147922" watchObservedRunningTime="2026-09-06 01:06:08.954800955 +0000 UTC m=+931.284159791"
Sep 06 01:06:12.422695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:12.422600    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:15.669459 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:15.669405    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:17.851616 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.851579    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"]
Sep 06 01:06:17.869369 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.869336    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:17.935049 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.935000    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 01:06:17.935248 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.935083    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 01:06:17.966225 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.966180    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:17.966535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.966489    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:17.979170 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.979133    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"]
Sep 06 01:06:17.982371 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.982341    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:17.983096 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.983071    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:17.985681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.984805    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:06:17.985681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.985055    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:06:17.989596 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:17.989572    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"]
Sep 06 01:06:18.019315 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.019274    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:06:18.024091 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.024053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e528ed6-027c-43ae-89f0-9a192ed73aab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-639343-lpqz6\" (UID: \"5e528ed6-027c-43ae-89f0-9a192ed73aab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:18.125084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.124980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e528ed6-027c-43ae-89f0-9a192ed73aab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-639343-lpqz6\" (UID: \"5e528ed6-027c-43ae-89f0-9a192ed73aab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:18.125829 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.125800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e528ed6-027c-43ae-89f0-9a192ed73aab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-639343-lpqz6\" (UID: \"5e528ed6-027c-43ae-89f0-9a192ed73aab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:18.245491 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.245442    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:18.297432 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.297405    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:18.571088 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:18.571058    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"]
Sep 06 01:06:19.012261 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:19.012221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6" event={"ID":"5e528ed6-027c-43ae-89f0-9a192ed73aab","Type":"ContainerStarted","Data":"2aadc277f1fdf42dd55d270974481758bd022227ff6806354f246e0e6bffd4ae"}
Sep 06 01:06:19.012734 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:19.012279    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6" event={"ID":"5e528ed6-027c-43ae-89f0-9a192ed73aab","Type":"ContainerStarted","Data":"753428acc94fbd02de792d152632e178545606b1cfdd399ab2c2dce26eacfb6a"}
Sep 06 01:06:19.030069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:19.029788    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6" podStartSLOduration=2.02976872 podStartE2EDuration="2.02976872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:19.029552915 +0000 UTC m=+941.358911753" watchObservedRunningTime="2026-09-06 01:06:19.02976872 +0000 UTC m=+941.359127558"
Sep 06 01:06:19.937542 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:19.937511    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434d572b_9b25_4751_b01b_f2faee69df80.slice/crio-conmon-853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42.scope\": RecentStats: unable to find data in memory cache]"
Sep 06 01:06:20.019534 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.019506    2575 generic.go:356] "Generic (PLEG): container finished" podID="5e528ed6-027c-43ae-89f0-9a192ed73aab" containerID="2aadc277f1fdf42dd55d270974481758bd022227ff6806354f246e0e6bffd4ae" exitCode=0
Sep 06 01:06:20.019928 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.019590    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6" event={"ID":"5e528ed6-027c-43ae-89f0-9a192ed73aab","Type":"ContainerDied","Data":"2aadc277f1fdf42dd55d270974481758bd022227ff6806354f246e0e6bffd4ae"}
Sep 06 01:06:20.021165 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.021147    2575 generic.go:356] "Generic (PLEG): container finished" podID="434d572b-9b25-4751-b01b-f2faee69df80" containerID="853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42" exitCode=0
Sep 06 01:06:20.021250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.021220    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" event={"ID":"434d572b-9b25-4751-b01b-f2faee69df80","Type":"ContainerDied","Data":"853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"}
Sep 06 01:06:20.021638 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.021625    2575 scope.go:117] "RemoveContainer" containerID="853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"
Sep 06 01:06:20.022733 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.022708    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:20.022805 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.022779    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:20.022843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:20.022815    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:21.028939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.028903    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" event={"ID":"434d572b-9b25-4751-b01b-f2faee69df80","Type":"ContainerStarted","Data":"d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1"}
Sep 06 01:06:21.143241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.143210    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:21.143452 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.143430    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 06 01:06:21.143509 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.143480    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 06 01:06:21.205238 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.205205    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:21.272899 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.272288    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e528ed6-027c-43ae-89f0-9a192ed73aab-maas-api-service-ca\") pod \"5e528ed6-027c-43ae-89f0-9a192ed73aab\" (UID: \"5e528ed6-027c-43ae-89f0-9a192ed73aab\") "
Sep 06 01:06:21.276875 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.275407    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e528ed6-027c-43ae-89f0-9a192ed73aab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e528ed6-027c-43ae-89f0-9a192ed73aab" (UID: "5e528ed6-027c-43ae-89f0-9a192ed73aab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:21.374422 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:21.374319    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e528ed6-027c-43ae-89f0-9a192ed73aab-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:22.035953 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:22.035919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6" event={"ID":"5e528ed6-027c-43ae-89f0-9a192ed73aab","Type":"ContainerDied","Data":"753428acc94fbd02de792d152632e178545606b1cfdd399ab2c2dce26eacfb6a"}
Sep 06 01:06:22.035953 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:22.035947    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"
Sep 06 01:06:22.036415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:22.035988    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="753428acc94fbd02de792d152632e178545606b1cfdd399ab2c2dce26eacfb6a"
Sep 06 01:06:22.143559 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:22.143520    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 06 01:06:22.143742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:22.143578    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 06 01:06:22.565623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:22.565507    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:23.143442 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:23.143408    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 06 01:06:23.143789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:23.143463    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 06 01:06:24.143751 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:24.143712    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 06 01:06:24.144160 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:24.143769    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 06 01:06:24.298653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:24.298616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:24.398653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:24.398579    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:25.143881 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:25.143826    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 06 01:06:25.144272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:25.143905    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 06 01:06:26.143825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:26.143792    2575 patch_prober.go:28] interesting pod/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 06 01:06:26.144029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:26.143875    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 06 01:06:27.146917 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.146887    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:27.147303 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.147183    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:27.148035 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.148018    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:27.958570 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.958536    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"]
Sep 06 01:06:27.958925 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.958912    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e528ed6-027c-43ae-89f0-9a192ed73aab" containerName="revoke-keys"
Sep 06 01:06:27.958977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.958926    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e528ed6-027c-43ae-89f0-9a192ed73aab" containerName="revoke-keys"
Sep 06 01:06:27.959017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.958995    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e528ed6-027c-43ae-89f0-9a192ed73aab" containerName="revoke-keys"
Sep 06 01:06:27.961096 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.961080    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:27.963396 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.963363    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:06:27.963533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.963442    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:06:27.970222 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:27.970192    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"]
Sep 06 01:06:28.039051 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.039018    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2\" (UID: \"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:28.139615 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.139585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2\" (UID: \"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:28.140256 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.140230    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2\" (UID: \"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:28.273614 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.273551    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:28.544788 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.544762    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"]
Sep 06 01:06:28.547093 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:28.547068    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d1e7a9_a1a6_4109_ae0a_5acf532f16fd.slice/crio-db2a7df2ad4bca3e768fdc8390fa86a6269e40472ccd6d6678e03eb5f7490396 WatchSource:0}: Error finding container db2a7df2ad4bca3e768fdc8390fa86a6269e40472ccd6d6678e03eb5f7490396: Status 404 returned error can't find the container with id db2a7df2ad4bca3e768fdc8390fa86a6269e40472ccd6d6678e03eb5f7490396
Sep 06 01:06:28.686144 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.686109    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:28.722443 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.722405    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"]
Sep 06 01:06:28.722642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.722601    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b" podUID="e0928735-2e49-4844-823e-fabc3484c85a" containerName="payload-processing" containerID="cri-o://244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410" gracePeriod=30
Sep 06 01:06:28.731371 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.731349    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"]
Sep 06 01:06:28.731599 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.731572    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c" podUID="0c7d98b3-d026-4467-88fa-f4af9c08971e" containerName="payload-pre-processing" containerID="cri-o://623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff" gracePeriod=30
Sep 06 01:06:28.769546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.769522    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:28.834484 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.834459    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:28.935065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.935036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:28.944663 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.944632    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"]
Sep 06 01:06:28.950295 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.950270    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-639343-lpqz6"]
Sep 06 01:06:28.983677 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.983655    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:06:28.993287 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:28.993267    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:06:29.016090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.016063    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:29.048664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.048633    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qlgq\" (UniqueName: \"kubernetes.io/projected/e0928735-2e49-4844-823e-fabc3484c85a-kube-api-access-6qlgq\") pod \"e0928735-2e49-4844-823e-fabc3484c85a\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") "
Sep 06 01:06:29.048823 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.048711    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c7d98b3-d026-4467-88fa-f4af9c08971e-plugins-config-volume\") pod \"0c7d98b3-d026-4467-88fa-f4af9c08971e\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") "
Sep 06 01:06:29.048925 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.048829    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0928735-2e49-4844-823e-fabc3484c85a-plugins-config-volume\") pod \"e0928735-2e49-4844-823e-fabc3484c85a\" (UID: \"e0928735-2e49-4844-823e-fabc3484c85a\") "
Sep 06 01:06:29.048925 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.048896    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4hrv\" (UniqueName: \"kubernetes.io/projected/0c7d98b3-d026-4467-88fa-f4af9c08971e-kube-api-access-q4hrv\") pod \"0c7d98b3-d026-4467-88fa-f4af9c08971e\" (UID: \"0c7d98b3-d026-4467-88fa-f4af9c08971e\") "
Sep 06 01:06:29.049203 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.049165    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c7d98b3-d026-4467-88fa-f4af9c08971e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c7d98b3-d026-4467-88fa-f4af9c08971e" (UID: "0c7d98b3-d026-4467-88fa-f4af9c08971e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:29.049203 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.049182    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0928735-2e49-4844-823e-fabc3484c85a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0928735-2e49-4844-823e-fabc3484c85a" (UID: "e0928735-2e49-4844-823e-fabc3484c85a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:29.049349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.049275    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0928735-2e49-4844-823e-fabc3484c85a-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:29.049349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.049289    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c7d98b3-d026-4467-88fa-f4af9c08971e-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:29.051012 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.050993    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0928735-2e49-4844-823e-fabc3484c85a-kube-api-access-6qlgq" (OuterVolumeSpecName: "kube-api-access-6qlgq") pod "e0928735-2e49-4844-823e-fabc3484c85a" (UID: "e0928735-2e49-4844-823e-fabc3484c85a"). InnerVolumeSpecName "kube-api-access-6qlgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:29.051217 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.051199    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c7d98b3-d026-4467-88fa-f4af9c08971e-kube-api-access-q4hrv" (OuterVolumeSpecName: "kube-api-access-q4hrv") pod "0c7d98b3-d026-4467-88fa-f4af9c08971e" (UID: "0c7d98b3-d026-4467-88fa-f4af9c08971e"). InnerVolumeSpecName "kube-api-access-q4hrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:29.107057 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.107017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2" event={"ID":"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd","Type":"ContainerStarted","Data":"fad9d60758021778c1bf3e0ebe8927d3c3dd70f14ed9d0c294142d7e6c6970e7"}
Sep 06 01:06:29.107255 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.107077    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2" event={"ID":"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd","Type":"ContainerStarted","Data":"db2a7df2ad4bca3e768fdc8390fa86a6269e40472ccd6d6678e03eb5f7490396"}
Sep 06 01:06:29.108379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.108347    2575 generic.go:356] "Generic (PLEG): container finished" podID="0c7d98b3-d026-4467-88fa-f4af9c08971e" containerID="623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff" exitCode=0
Sep 06 01:06:29.108379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.108378    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"
Sep 06 01:06:29.108587 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.108425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c" event={"ID":"0c7d98b3-d026-4467-88fa-f4af9c08971e","Type":"ContainerDied","Data":"623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff"}
Sep 06 01:06:29.108587 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.108467    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c" event={"ID":"0c7d98b3-d026-4467-88fa-f4af9c08971e","Type":"ContainerDied","Data":"c357c61c989a7d09d9547dcbc0e8c2b958f3cb3e63963e8d137fe8ece37c9d7d"}
Sep 06 01:06:29.108587 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.108490    2575 scope.go:117] "RemoveContainer" containerID="623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff"
Sep 06 01:06:29.110277 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.110251    2575 generic.go:356] "Generic (PLEG): container finished" podID="e0928735-2e49-4844-823e-fabc3484c85a" containerID="244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410" exitCode=0
Sep 06 01:06:29.110404 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.110286    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b" event={"ID":"e0928735-2e49-4844-823e-fabc3484c85a","Type":"ContainerDied","Data":"244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410"}
Sep 06 01:06:29.110404 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.110317    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"
Sep 06 01:06:29.110404 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.110323    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b" event={"ID":"e0928735-2e49-4844-823e-fabc3484c85a","Type":"ContainerDied","Data":"bcf847e747611ecfa99062d5b139cbcbe14033e331442561886bc9a626799b63"}
Sep 06 01:06:29.121274 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.121255    2575 scope.go:117] "RemoveContainer" containerID="623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff"
Sep 06 01:06:29.121563 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:29.121539    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff\": container with ID starting with 623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff not found: ID does not exist" containerID="623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff"
Sep 06 01:06:29.121625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.121571    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff"} err="failed to get container status \"623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff\": rpc error: code = NotFound desc = could not find container \"623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff\": container with ID starting with 623e2bc11b89fba14a5a3d13690cbdc68242a13caf2c52ecf45a0fa5001dd2ff not found: ID does not exist"
Sep 06 01:06:29.121625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.121596    2575 scope.go:117] "RemoveContainer" containerID="244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410"
Sep 06 01:06:29.127151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.125627    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2" podStartSLOduration=2.125611376 podStartE2EDuration="2.125611376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:29.122518636 +0000 UTC m=+951.451877472" watchObservedRunningTime="2026-09-06 01:06:29.125611376 +0000 UTC m=+951.454970212"
Sep 06 01:06:29.135916 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.135897    2575 scope.go:117] "RemoveContainer" containerID="244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410"
Sep 06 01:06:29.136215 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:29.136191    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410\": container with ID starting with 244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410 not found: ID does not exist" containerID="244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410"
Sep 06 01:06:29.136279 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.136225    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410"} err="failed to get container status \"244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410\": rpc error: code = NotFound desc = could not find container \"244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410\": container with ID starting with 244d85e3235d20b9fa52f15973c943425e9c84b8f9855f75a6e9383c9e1fb410 not found: ID does not exist"
Sep 06 01:06:29.137594 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.137575    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"]
Sep 06 01:06:29.142377 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.142356    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-639343-7cb59bd6df-jvw8b"]
Sep 06 01:06:29.150121 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.150100    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4hrv\" (UniqueName: \"kubernetes.io/projected/0c7d98b3-d026-4467-88fa-f4af9c08971e-kube-api-access-q4hrv\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:29.150121 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.150122    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qlgq\" (UniqueName: \"kubernetes.io/projected/e0928735-2e49-4844-823e-fabc3484c85a-kube-api-access-6qlgq\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:29.153711 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.153690    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"]
Sep 06 01:06:29.159669 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.159637    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-639343-54d6bb9d66-pqh5c"]
Sep 06 01:06:29.291757 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:29.291721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:30.117590 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:30.117553    2575 generic.go:356] "Generic (PLEG): container finished" podID="91d1e7a9-a1a6-4109-ae0a-5acf532f16fd" containerID="fad9d60758021778c1bf3e0ebe8927d3c3dd70f14ed9d0c294142d7e6c6970e7" exitCode=0
Sep 06 01:06:30.117786 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:30.117639    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2" event={"ID":"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd","Type":"ContainerDied","Data":"fad9d60758021778c1bf3e0ebe8927d3c3dd70f14ed9d0c294142d7e6c6970e7"}
Sep 06 01:06:30.201731 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:30.201701    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c7d98b3-d026-4467-88fa-f4af9c08971e" path="/var/lib/kubelet/pods/0c7d98b3-d026-4467-88fa-f4af9c08971e/volumes"
Sep 06 01:06:30.202102 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:30.202087    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e528ed6-027c-43ae-89f0-9a192ed73aab" path="/var/lib/kubelet/pods/5e528ed6-027c-43ae-89f0-9a192ed73aab/volumes"
Sep 06 01:06:30.202401 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:30.202389    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0928735-2e49-4844-823e-fabc3484c85a" path="/var/lib/kubelet/pods/e0928735-2e49-4844-823e-fabc3484c85a/volumes"
Sep 06 01:06:31.273973 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:31.273944    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:31.371270 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:31.371242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd-maas-api-service-ca\") pod \"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd\" (UID: \"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd\") "
Sep 06 01:06:31.371624 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:31.371597    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91d1e7a9-a1a6-4109-ae0a-5acf532f16fd" (UID: "91d1e7a9-a1a6-4109-ae0a-5acf532f16fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:31.472308 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:31.472284    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:32.132877 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:32.132828    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"
Sep 06 01:06:32.132877 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:32.132868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2" event={"ID":"91d1e7a9-a1a6-4109-ae0a-5acf532f16fd","Type":"ContainerDied","Data":"db2a7df2ad4bca3e768fdc8390fa86a6269e40472ccd6d6678e03eb5f7490396"}
Sep 06 01:06:32.133128 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:32.132909    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db2a7df2ad4bca3e768fdc8390fa86a6269e40472ccd6d6678e03eb5f7490396"
Sep 06 01:06:32.876382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:32.876335    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44" podUID="0a2e7306-6296-4f26-890f-e6556f188aad" containerName="main" containerID="cri-o://89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df" gracePeriod=45
Sep 06 01:06:33.122231 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.122210    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:06:33.141378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.141307    2575 generic.go:356] "Generic (PLEG): container finished" podID="0a2e7306-6296-4f26-890f-e6556f188aad" containerID="89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df" exitCode=0
Sep 06 01:06:33.141378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.141350    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"
Sep 06 01:06:33.141533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.141347    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44" event={"ID":"0a2e7306-6296-4f26-890f-e6556f188aad","Type":"ContainerDied","Data":"89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df"}
Sep 06 01:06:33.141533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.141441    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44" event={"ID":"0a2e7306-6296-4f26-890f-e6556f188aad","Type":"ContainerDied","Data":"0938ad0d6c5f4d3bae55c3f74767d0e57a417f079be7e97b8da56cd49a561ef7"}
Sep 06 01:06:33.141533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.141474    2575 scope.go:117] "RemoveContainer" containerID="89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df"
Sep 06 01:06:33.152989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.152971    2575 scope.go:117] "RemoveContainer" containerID="89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df"
Sep 06 01:06:33.153258 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:33.153230    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df\": container with ID starting with 89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df not found: ID does not exist" containerID="89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df"
Sep 06 01:06:33.153304 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.153274    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df"} err="failed to get container status \"89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df\": rpc error: code = NotFound desc = could not find container \"89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df\": container with ID starting with 89f0e67a2da4138402011e90d9229856102ba6dd3c09bf770912bd9407e3c5df not found: ID does not exist"
Sep 06 01:06:33.188153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188126    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-tmp-dir\") pod \"0a2e7306-6296-4f26-890f-e6556f188aad\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") "
Sep 06 01:06:33.188267 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188188    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2e7306-6296-4f26-890f-e6556f188aad-tls-certs\") pod \"0a2e7306-6296-4f26-890f-e6556f188aad\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") "
Sep 06 01:06:33.188267 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188216    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4vt8\" (UniqueName: \"kubernetes.io/projected/0a2e7306-6296-4f26-890f-e6556f188aad-kube-api-access-l4vt8\") pod \"0a2e7306-6296-4f26-890f-e6556f188aad\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") "
Sep 06 01:06:33.188267 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188232    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-home\") pod \"0a2e7306-6296-4f26-890f-e6556f188aad\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") "
Sep 06 01:06:33.188267 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188264    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-dshm\") pod \"0a2e7306-6296-4f26-890f-e6556f188aad\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") "
Sep 06 01:06:33.188479 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188286    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-model-cache\") pod \"0a2e7306-6296-4f26-890f-e6556f188aad\" (UID: \"0a2e7306-6296-4f26-890f-e6556f188aad\") "
Sep 06 01:06:33.188479 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188410    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a2e7306-6296-4f26-890f-e6556f188aad" (UID: "0a2e7306-6296-4f26-890f-e6556f188aad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:33.188580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188537    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-home" (OuterVolumeSpecName: "home") pod "0a2e7306-6296-4f26-890f-e6556f188aad" (UID: "0a2e7306-6296-4f26-890f-e6556f188aad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:33.188629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188580    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a2e7306-6296-4f26-890f-e6556f188aad" (UID: "0a2e7306-6296-4f26-890f-e6556f188aad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:33.188629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188620    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:33.188737 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.188632    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:33.190648 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.190624    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2e7306-6296-4f26-890f-e6556f188aad-kube-api-access-l4vt8" (OuterVolumeSpecName: "kube-api-access-l4vt8") pod "0a2e7306-6296-4f26-890f-e6556f188aad" (UID: "0a2e7306-6296-4f26-890f-e6556f188aad"). InnerVolumeSpecName "kube-api-access-l4vt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:33.190648 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.190643    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2e7306-6296-4f26-890f-e6556f188aad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a2e7306-6296-4f26-890f-e6556f188aad" (UID: "0a2e7306-6296-4f26-890f-e6556f188aad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:06:33.190768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.190651    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-dshm" (OuterVolumeSpecName: "dshm") pod "0a2e7306-6296-4f26-890f-e6556f188aad" (UID: "0a2e7306-6296-4f26-890f-e6556f188aad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:33.289911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.289881    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2e7306-6296-4f26-890f-e6556f188aad-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:33.289911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.289905    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4vt8\" (UniqueName: \"kubernetes.io/projected/0a2e7306-6296-4f26-890f-e6556f188aad-kube-api-access-l4vt8\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:33.289911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.289916    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:33.290129 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.289925    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a2e7306-6296-4f26-890f-e6556f188aad-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:33.466707 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.466663    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"]
Sep 06 01:06:33.468964 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.468945    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-639343/autores-model-639343-kserve-c95947bb7-6hz44"]
Sep 06 01:06:33.963392 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:33.963349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:34.039623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.039591    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:34.106781 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.106680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:34.157765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.157732    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"]
Sep 06 01:06:34.163783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.163744    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6cee6e60-wn2m2"]
Sep 06 01:06:34.169741 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.169714    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:34.202741 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.202711    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a2e7306-6296-4f26-890f-e6556f188aad" path="/var/lib/kubelet/pods/0a2e7306-6296-4f26-890f-e6556f188aad/volumes"
Sep 06 01:06:34.203083 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:34.203069    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91d1e7a9-a1a6-4109-ae0a-5acf532f16fd" path="/var/lib/kubelet/pods/91d1e7a9-a1a6-4109-ae0a-5acf532f16fd/volumes"
Sep 06 01:06:35.380372 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:35.380333    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"]
Sep 06 01:06:35.380778 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:35.380547    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy" containerID="cri-o://d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1" gracePeriod=30
Sep 06 01:06:35.385494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:35.385469    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:35.859425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:35.859360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:36.138454 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138384    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"]
Sep 06 01:06:36.138728 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138717    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c7d98b3-d026-4467-88fa-f4af9c08971e" containerName="payload-pre-processing"
Sep 06 01:06:36.138771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138730    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c7d98b3-d026-4467-88fa-f4af9c08971e" containerName="payload-pre-processing"
Sep 06 01:06:36.138771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138747    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0928735-2e49-4844-823e-fabc3484c85a" containerName="payload-processing"
Sep 06 01:06:36.138771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138752    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0928735-2e49-4844-823e-fabc3484c85a" containerName="payload-processing"
Sep 06 01:06:36.138771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138763    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a2e7306-6296-4f26-890f-e6556f188aad" containerName="main"
Sep 06 01:06:36.138771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138769    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2e7306-6296-4f26-890f-e6556f188aad" containerName="main"
Sep 06 01:06:36.138941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138783    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91d1e7a9-a1a6-4109-ae0a-5acf532f16fd" containerName="revoke-keys"
Sep 06 01:06:36.138941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138788    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="91d1e7a9-a1a6-4109-ae0a-5acf532f16fd" containerName="revoke-keys"
Sep 06 01:06:36.138941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138838    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0928735-2e49-4844-823e-fabc3484c85a" containerName="payload-processing"
Sep 06 01:06:36.138941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138845    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a2e7306-6296-4f26-890f-e6556f188aad" containerName="main"
Sep 06 01:06:36.138941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138871    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c7d98b3-d026-4467-88fa-f4af9c08971e" containerName="payload-pre-processing"
Sep 06 01:06:36.138941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.138880    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="91d1e7a9-a1a6-4109-ae0a-5acf532f16fd" containerName="revoke-keys"
Sep 06 01:06:36.141473 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.141457    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.143796 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.143775    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8a9d2066-openshift-default-dockercfg-b2xs6\""
Sep 06 01:06:36.153649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.153628    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"]
Sep 06 01:06:36.164813 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.164785    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:36.217795 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.217769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-credential-socket\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.217994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.217801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-istio-token\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.217994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.217834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-socket\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.217994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.217914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v454\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-kube-api-access-5v454\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.217994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.217936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-envoy\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.218201 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.218014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2b8040-3024-4378-88c8-41130ab222cd-istio-podinfo\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.218201 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.218090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2b8040-3024-4378-88c8-41130ab222cd-istiod-ca-cert\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.218201 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.218122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-certs\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.218305 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.218199    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-data\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319468 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319437    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2b8040-3024-4378-88c8-41130ab222cd-istiod-ca-cert\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319468 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319474    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-certs\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319512    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-data\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-credential-socket\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-istio-token\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319595    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-socket\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319761    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v454\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-kube-api-access-5v454\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-envoy\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2b8040-3024-4378-88c8-41130ab222cd-istio-podinfo\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.319983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319944    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-credential-socket\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.320193 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.319983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-certs\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.320193 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.320024    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-data\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.320289 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.320211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-socket\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.320289 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.320250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2b8040-3024-4378-88c8-41130ab222cd-istiod-ca-cert\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.322285 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.322256    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-envoy\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.322496 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.322477    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2b8040-3024-4378-88c8-41130ab222cd-istio-podinfo\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.328677 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.328651    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v454\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-kube-api-access-5v454\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.329488 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.329466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-istio-token\") pod \"e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") " pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.370057 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.370034    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:36.452592 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.452566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:36.470296 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.470261    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:36.533641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.533613    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:36.693811 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.693775    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"]
Sep 06 01:06:36.694068 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.694013    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" containerID="cri-o://e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7" gracePeriod=30
Sep 06 01:06:36.719124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.719031    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:36.732078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.732051    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"]
Sep 06 01:06:36.733664 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:36.733639    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2b8040_3024_4378_88c8_41130ab222cd.slice/crio-a01460e572ab6daac7fa5872e15d2cb86d53b3bd9cda194b7a6eff6cd74842b3 WatchSource:0}: Error finding container a01460e572ab6daac7fa5872e15d2cb86d53b3bd9cda194b7a6eff6cd74842b3: Status 404 returned error can't find the container with id a01460e572ab6daac7fa5872e15d2cb86d53b3bd9cda194b7a6eff6cd74842b3
Sep 06 01:06:36.735976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.735944    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:36.736080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.736016    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:36.736148 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:36.736077    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:06:37.159844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.159749    2575 patch_prober.go:28] interesting pod/e2e-autores-639343-openshift-default-684844c85d-b8f2q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 01:06:37.159844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.159807    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 01:06:37.168947 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.168918    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" event={"ID":"ed2b8040-3024-4378-88c8-41130ab222cd","Type":"ContainerStarted","Data":"fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"}
Sep 06 01:06:37.169116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.168967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" event={"ID":"ed2b8040-3024-4378-88c8-41130ab222cd","Type":"ContainerStarted","Data":"a01460e572ab6daac7fa5872e15d2cb86d53b3bd9cda194b7a6eff6cd74842b3"}
Sep 06 01:06:37.175919 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.175894    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:37.190841 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.190793    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podStartSLOduration=1.190779679 podStartE2EDuration="1.190779679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:37.187714845 +0000 UTC m=+959.517073684" watchObservedRunningTime="2026-09-06 01:06:37.190779679 +0000 UTC m=+959.520138514"
Sep 06 01:06:37.453756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.453725    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:37.458580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.458552    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:37.467095 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.467070    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:37.632912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:37.632872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:38.176144 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:38.176116    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:38.177113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:38.177091    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:06:38.581010 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:38.580957    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:38.670353 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:38.670325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:39.952309 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:39.952259    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.054741 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.054705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.261167 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.261091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.353740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.353710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.459769 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.459737    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.633829 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.633803    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:40.746490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.746462    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.763554 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763524    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-certs\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.763760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763586    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-credential-socket\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.763760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763622    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d572b-9b25-4751-b01b-f2faee69df80-istio-podinfo\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.763760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763655    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d572b-9b25-4751-b01b-f2faee69df80-istiod-ca-cert\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.763760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763682    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7msp\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-kube-api-access-k7msp\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.763760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763718    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-socket\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.764070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763764    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-envoy\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.764070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763781    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-istio-token\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.764070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-data\") pod \"434d572b-9b25-4751-b01b-f2faee69df80\" (UID: \"434d572b-9b25-4751-b01b-f2faee69df80\") "
Sep 06 01:06:40.764070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.763839    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:40.764253 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.764198    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.764493 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.764452    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-data" (OuterVolumeSpecName: "istio-data") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:40.764493 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.764483    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:40.764720 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.764703    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:40.764824 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.764803    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434d572b-9b25-4751-b01b-f2faee69df80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:40.766991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.766962    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-istio-token" (OuterVolumeSpecName: "istio-token") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:40.767133 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.767027    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-kube-api-access-k7msp" (OuterVolumeSpecName: "kube-api-access-k7msp") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "kube-api-access-k7msp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:40.767437 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.767415    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/434d572b-9b25-4751-b01b-f2faee69df80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:06:40.767437 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.767424    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "434d572b-9b25-4751-b01b-f2faee69df80" (UID: "434d572b-9b25-4751-b01b-f2faee69df80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:40.845559 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.845451    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:40.865666 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865641    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865666 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865664    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d572b-9b25-4751-b01b-f2faee69df80-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865674    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d572b-9b25-4751-b01b-f2faee69df80-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865683    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7msp\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-kube-api-access-k7msp\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865692    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865700    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865707    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d572b-9b25-4751-b01b-f2faee69df80-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.865826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.865714    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d572b-9b25-4751-b01b-f2faee69df80-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:40.882278 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882256    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"]
Sep 06 01:06:40.882636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882624    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy"
Sep 06 01:06:40.882701 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882638    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy"
Sep 06 01:06:40.882701 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882653    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy"
Sep 06 01:06:40.882701 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882658    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy"
Sep 06 01:06:40.882808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882714    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy"
Sep 06 01:06:40.882808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.882724    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="434d572b-9b25-4751-b01b-f2faee69df80" containerName="istio-proxy"
Sep 06 01:06:40.884783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.884770    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:40.887981 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.887963    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-8a9d2066-dockercfg-v2qbm\""
Sep 06 01:06:40.889264 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.889247    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-8a9d2066\""
Sep 06 01:06:40.901716 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.901697    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"]
Sep 06 01:06:40.964299 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.964270    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"]
Sep 06 01:06:40.966582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.966566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:40.981658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:40.981638    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"]
Sep 06 01:06:41.067345 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.067309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jkpr\" (UniqueName: \"kubernetes.io/projected/b023a635-0eff-4b81-8844-64120111a728-kube-api-access-8jkpr\") pod \"payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") " pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.067517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.067395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977adebd-9ca3-4795-8d5a-7e54f64cc619-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.067517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.067428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc2jm\" (UniqueName: \"kubernetes.io/projected/977adebd-9ca3-4795-8d5a-7e54f64cc619-kube-api-access-cc2jm\") pod \"payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.067517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.067502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b023a635-0eff-4b81-8844-64120111a728-plugins-config-volume\") pod \"payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") " pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.168874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.168764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jkpr\" (UniqueName: \"kubernetes.io/projected/b023a635-0eff-4b81-8844-64120111a728-kube-api-access-8jkpr\") pod \"payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") " pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.168874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.168822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977adebd-9ca3-4795-8d5a-7e54f64cc619-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.168874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.168840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc2jm\" (UniqueName: \"kubernetes.io/projected/977adebd-9ca3-4795-8d5a-7e54f64cc619-kube-api-access-cc2jm\") pod \"payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.169149 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.168930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b023a635-0eff-4b81-8844-64120111a728-plugins-config-volume\") pod \"payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") " pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.169498 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.169478    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977adebd-9ca3-4795-8d5a-7e54f64cc619-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.169597 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.169580    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b023a635-0eff-4b81-8844-64120111a728-plugins-config-volume\") pod \"payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") " pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.178197 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.178175    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jkpr\" (UniqueName: \"kubernetes.io/projected/b023a635-0eff-4b81-8844-64120111a728-kube-api-access-8jkpr\") pod \"payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") " pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.178276 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.178173    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc2jm\" (UniqueName: \"kubernetes.io/projected/977adebd-9ca3-4795-8d5a-7e54f64cc619-kube-api-access-cc2jm\") pod \"payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.194921 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.194897    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:41.195033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.194935    2575 generic.go:356] "Generic (PLEG): container finished" podID="434d572b-9b25-4751-b01b-f2faee69df80" containerID="d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1" exitCode=0
Sep 06 01:06:41.195033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.195020    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"
Sep 06 01:06:41.195124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.195054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" event={"ID":"434d572b-9b25-4751-b01b-f2faee69df80","Type":"ContainerDied","Data":"d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1"}
Sep 06 01:06:41.195124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.195109    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h" event={"ID":"434d572b-9b25-4751-b01b-f2faee69df80","Type":"ContainerDied","Data":"48c08739bfb0e4c35c78b7fcc5059727589546b85029ac94ef5f4f1b42deb527"}
Sep 06 01:06:41.195200 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.195132    2575 scope.go:117] "RemoveContainer" containerID="d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1"
Sep 06 01:06:41.205998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.205982    2575 scope.go:117] "RemoveContainer" containerID="853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"
Sep 06 01:06:41.214416 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.214399    2575 scope.go:117] "RemoveContainer" containerID="d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1"
Sep 06 01:06:41.214716 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:41.214697    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1\": container with ID starting with d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1 not found: ID does not exist" containerID="d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1"
Sep 06 01:06:41.214779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.214724    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1"} err="failed to get container status \"d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1\": rpc error: code = NotFound desc = could not find container \"d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1\": container with ID starting with d06d5f886212f5faed87efbac4346b10d041d4d11fd3979c24a06f536e5b7fe1 not found: ID does not exist"
Sep 06 01:06:41.214779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.214740    2575 scope.go:117] "RemoveContainer" containerID="853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"
Sep 06 01:06:41.215064 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:41.215046    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42\": container with ID starting with 853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42 not found: ID does not exist" containerID="853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"
Sep 06 01:06:41.215119 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.215070    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42"} err="failed to get container status \"853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42\": rpc error: code = NotFound desc = could not find container \"853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42\": container with ID starting with 853904eff1f267b8fe33d1e723e0bbb5157f9c6603d9220dd527873516151e42 not found: ID does not exist"
Sep 06 01:06:41.223272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.223247    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"]
Sep 06 01:06:41.229706 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.229676    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6cee6e60-openshift-default-5b7d6d58f6-k9f8h"]
Sep 06 01:06:41.275940 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.275915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:41.475581 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.475556    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"]
Sep 06 01:06:41.477336 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:41.477311    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb023a635_0eff_4b81_8844_64120111a728.slice/crio-e1bcbce321e344d43820c16b98830780e37b6919a41b64d4e094336b34536e04 WatchSource:0}: Error finding container e1bcbce321e344d43820c16b98830780e37b6919a41b64d4e094336b34536e04: Status 404 returned error can't find the container with id e1bcbce321e344d43820c16b98830780e37b6919a41b64d4e094336b34536e04
Sep 06 01:06:41.555049 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.555018    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"]
Sep 06 01:06:41.556156 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:41.556126    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977adebd_9ca3_4795_8d5a_7e54f64cc619.slice/crio-58ff0ed03ccd5fd13c0d6da20dfe918586a42b37632fc2289531ecaa987533a0 WatchSource:0}: Error finding container 58ff0ed03ccd5fd13c0d6da20dfe918586a42b37632fc2289531ecaa987533a0: Status 404 returned error can't find the container with id 58ff0ed03ccd5fd13c0d6da20dfe918586a42b37632fc2289531ecaa987533a0
Sep 06 01:06:41.942563 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:41.942540    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:06:42.069082 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.068963    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:42.076520 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076497    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-data\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076540    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfw5d\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-kube-api-access-gfw5d\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076623    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-credential-socket\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076683    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-socket\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076735    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-certs\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076759    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-envoy\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076784    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-podinfo\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076779    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:42.076943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076843    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istiod-ca-cert\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.076943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076935    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-token\") pod \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\" (UID: \"d7a8e0eb-fdf5-4044-8f68-61f1f0791496\") "
Sep 06 01:06:42.077136 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076944    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:42.077136 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.076969    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:42.077136 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.077113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:42.077295 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.077257    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:06:42.077361 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.077296    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.077361 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.077314    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.077361 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.077322    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.077361 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.077331    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.079129 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.079093    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:06:42.079598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.079573    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-kube-api-access-gfw5d" (OuterVolumeSpecName: "kube-api-access-gfw5d") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "kube-api-access-gfw5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:42.079598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.079579    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:06:42.079598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.079589    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7a8e0eb-fdf5-4044-8f68-61f1f0791496" (UID: "d7a8e0eb-fdf5-4044-8f68-61f1f0791496"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:06:42.151425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.151393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:42.178659 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.178631    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.178659 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.178658    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.178897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.178668    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.178897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.178675    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.178897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.178683    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfw5d\" (UniqueName: \"kubernetes.io/projected/d7a8e0eb-fdf5-4044-8f68-61f1f0791496-kube-api-access-gfw5d\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:06:42.203453 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.203421    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="434d572b-9b25-4751-b01b-f2faee69df80" path="/var/lib/kubelet/pods/434d572b-9b25-4751-b01b-f2faee69df80/volumes"
Sep 06 01:06:42.203960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.203938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk" event={"ID":"b023a635-0eff-4b81-8844-64120111a728","Type":"ContainerStarted","Data":"e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd"}
Sep 06 01:06:42.204099 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.203987    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:42.204099 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.204026    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk" event={"ID":"b023a635-0eff-4b81-8844-64120111a728","Type":"ContainerStarted","Data":"e1bcbce321e344d43820c16b98830780e37b6919a41b64d4e094336b34536e04"}
Sep 06 01:06:42.204412 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.204386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl" event={"ID":"977adebd-9ca3-4795-8d5a-7e54f64cc619","Type":"ContainerStarted","Data":"fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b"}
Sep 06 01:06:42.204547 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.204441    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:42.204547 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.204465    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl" event={"ID":"977adebd-9ca3-4795-8d5a-7e54f64cc619","Type":"ContainerStarted","Data":"58ff0ed03ccd5fd13c0d6da20dfe918586a42b37632fc2289531ecaa987533a0"}
Sep 06 01:06:42.206239 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.206217    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerID="e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7" exitCode=0
Sep 06 01:06:42.206321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.206247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" event={"ID":"d7a8e0eb-fdf5-4044-8f68-61f1f0791496","Type":"ContainerDied","Data":"e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7"}
Sep 06 01:06:42.206321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.206269    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"
Sep 06 01:06:42.206321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.206288    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q" event={"ID":"d7a8e0eb-fdf5-4044-8f68-61f1f0791496","Type":"ContainerDied","Data":"e669c327b26bb2548010d16965cefcca0c505b8fcd68374092a45448d0e47c85"}
Sep 06 01:06:42.206497 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.206323    2575 scope.go:117] "RemoveContainer" containerID="e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7"
Sep 06 01:06:42.217326 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.217311    2575 scope.go:117] "RemoveContainer" containerID="b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"
Sep 06 01:06:42.226203 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.226178    2575 scope.go:117] "RemoveContainer" containerID="e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7"
Sep 06 01:06:42.226486 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:42.226463    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7\": container with ID starting with e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7 not found: ID does not exist" containerID="e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7"
Sep 06 01:06:42.226546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.226498    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7"} err="failed to get container status \"e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7\": rpc error: code = NotFound desc = could not find container \"e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7\": container with ID starting with e7986d879dc48bac63cd9ebe6759534392d56a055bde8d35fd96dba3a33a40b7 not found: ID does not exist"
Sep 06 01:06:42.226546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.226521    2575 scope.go:117] "RemoveContainer" containerID="b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"
Sep 06 01:06:42.226805 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:06:42.226785    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4\": container with ID starting with b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4 not found: ID does not exist" containerID="b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"
Sep 06 01:06:42.226897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.226813    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4"} err="failed to get container status \"b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4\": rpc error: code = NotFound desc = could not find container \"b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4\": container with ID starting with b05489f759a0d7e32cb7e0da89a2a3e2bafcd3e96e830f0d7084f2ec58e75ca4 not found: ID does not exist"
Sep 06 01:06:42.227181 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.227148    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk" podStartSLOduration=2.227136776 podStartE2EDuration="2.227136776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:42.223893653 +0000 UTC m=+964.553252515" watchObservedRunningTime="2026-09-06 01:06:42.227136776 +0000 UTC m=+964.556495611"
Sep 06 01:06:42.243580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.243542    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"]
Sep 06 01:06:42.249733 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.249709    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-639343-openshift-default-684844c85d-b8f2q"]
Sep 06 01:06:42.274241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.274199    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl" podStartSLOduration=2.274186169 podStartE2EDuration="2.274186169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:42.273209665 +0000 UTC m=+964.602568527" watchObservedRunningTime="2026-09-06 01:06:42.274186169 +0000 UTC m=+964.603545005"
Sep 06 01:06:42.540971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.540938    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:42.718623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.718596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:42.814755 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.814724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:42.846158 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.846126    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"]
Sep 06 01:06:42.846487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.846474    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy"
Sep 06 01:06:42.846527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.846489    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy"
Sep 06 01:06:42.846527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.846503    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy"
Sep 06 01:06:42.846527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.846511    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy"
Sep 06 01:06:42.846618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.846572    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy"
Sep 06 01:06:42.849675 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.849659    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:42.853028 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.853008    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-1fc789-dockercfg-45btn\""
Sep 06 01:06:42.853122 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.853051    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-1fc789\""
Sep 06 01:06:42.861291 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.861266    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"]
Sep 06 01:06:42.938484 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.938456    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"]
Sep 06 01:06:42.938911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.938898    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" containerName="istio-proxy"
Sep 06 01:06:42.940943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.940926    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:42.953992 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.953970    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"]
Sep 06 01:06:42.985201 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.985123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c1d59-ac15-45fd-9191-af063349df66-plugins-config-volume\") pod \"payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") " pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:42.985201 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:42.985194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlcmj\" (UniqueName: \"kubernetes.io/projected/ca6c1d59-ac15-45fd-9191-af063349df66-kube-api-access-dlcmj\") pod \"payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") " pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:43.086322 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.086291    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c1d59-ac15-45fd-9191-af063349df66-plugins-config-volume\") pod \"payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") " pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:43.086768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.086358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjp9w\" (UniqueName: \"kubernetes.io/projected/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-kube-api-access-vjp9w\") pod \"payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.086768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.086385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlcmj\" (UniqueName: \"kubernetes.io/projected/ca6c1d59-ac15-45fd-9191-af063349df66-kube-api-access-dlcmj\") pod \"payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") " pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:43.086768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.086422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.086997 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.086977    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c1d59-ac15-45fd-9191-af063349df66-plugins-config-volume\") pod \"payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") " pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:43.095098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.095076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlcmj\" (UniqueName: \"kubernetes.io/projected/ca6c1d59-ac15-45fd-9191-af063349df66-kube-api-access-dlcmj\") pod \"payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") " pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:43.160529 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.160506    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:43.187846 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.187819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjp9w\" (UniqueName: \"kubernetes.io/projected/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-kube-api-access-vjp9w\") pod \"payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.188009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.187910    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.188540 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.188518    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.196213 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.196186    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjp9w\" (UniqueName: \"kubernetes.io/projected/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-kube-api-access-vjp9w\") pod \"payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.251355 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.251286    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:43.431782 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.431758    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"]
Sep 06 01:06:43.433351 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:43.433325    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6c1d59_ac15_45fd_9191_af063349df66.slice/crio-31095b414ca5ddda5d284716c43c4a5608a79c318348b10ca41bf55f4a46692c WatchSource:0}: Error finding container 31095b414ca5ddda5d284716c43c4a5608a79c318348b10ca41bf55f4a46692c: Status 404 returned error can't find the container with id 31095b414ca5ddda5d284716c43c4a5608a79c318348b10ca41bf55f4a46692c
Sep 06 01:06:43.526572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:43.526545    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"]
Sep 06 01:06:43.528261 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:43.528234    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ae230f_a9d9_4c8a_84f8_eddcbc017a9d.slice/crio-6a4651381b1ead003a4567013535bcbd8e374417d74e6b7a256199ab586e7162 WatchSource:0}: Error finding container 6a4651381b1ead003a4567013535bcbd8e374417d74e6b7a256199ab586e7162: Status 404 returned error can't find the container with id 6a4651381b1ead003a4567013535bcbd8e374417d74e6b7a256199ab586e7162
Sep 06 01:06:44.203834 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.203801    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7a8e0eb-fdf5-4044-8f68-61f1f0791496" path="/var/lib/kubelet/pods/d7a8e0eb-fdf5-4044-8f68-61f1f0791496/volumes"
Sep 06 01:06:44.222766 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.222739    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2" event={"ID":"ca6c1d59-ac15-45fd-9191-af063349df66","Type":"ContainerStarted","Data":"af77973b2a57879c7c845b3fbfa7cb53284a294317090fff5b795dd4e6dc8673"}
Sep 06 01:06:44.222921 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.222799    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:44.222921 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.222827    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2" event={"ID":"ca6c1d59-ac15-45fd-9191-af063349df66","Type":"ContainerStarted","Data":"31095b414ca5ddda5d284716c43c4a5608a79c318348b10ca41bf55f4a46692c"}
Sep 06 01:06:44.224349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.224327    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd" event={"ID":"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d","Type":"ContainerStarted","Data":"80be7dafc9c75a9d2674f9c7b55dda6092bf008b50d381a78f9d8c5b4b771783"}
Sep 06 01:06:44.224469 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.224372    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd" event={"ID":"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d","Type":"ContainerStarted","Data":"6a4651381b1ead003a4567013535bcbd8e374417d74e6b7a256199ab586e7162"}
Sep 06 01:06:44.224469 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.224402    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:44.248504 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.248455    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2" podStartSLOduration=2.24844205 podStartE2EDuration="2.24844205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:44.247029186 +0000 UTC m=+966.576388062" watchObservedRunningTime="2026-09-06 01:06:44.24844205 +0000 UTC m=+966.577800885"
Sep 06 01:06:44.265012 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:44.264970    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd" podStartSLOduration=2.264957071 podStartE2EDuration="2.264957071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:44.262476653 +0000 UTC m=+966.591835510" watchObservedRunningTime="2026-09-06 01:06:44.264957071 +0000 UTC m=+966.594315906"
Sep 06 01:06:46.152030 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.151995    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"]
Sep 06 01:06:46.154543 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.154527    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.157499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.157476    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8d5457\"/\"default-dockercfg-f5tks\""
Sep 06 01:06:46.157611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.157508    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8d5457\"/\"kube-root-ca.crt\""
Sep 06 01:06:46.157611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.157517    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8d5457\"/\"openshift-service-ca.crt\""
Sep 06 01:06:46.157611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.157482    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8d5457\"/\"test-model-8d5457-kserve-self-signed-certs\""
Sep 06 01:06:46.163249 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.163224    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"]
Sep 06 01:06:46.318062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.318014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-model-cache\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.318062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.318068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51c3d073-7877-4059-b1c8-e6f477af8436-tls-certs\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.318272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.318136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-home\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.318272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.318181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-dshm\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.318272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.318205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-tmp-dir\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.318272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.318242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84nss\" (UniqueName: \"kubernetes.io/projected/51c3d073-7877-4059-b1c8-e6f477af8436-kube-api-access-84nss\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419145 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419051    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-model-cache\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419145 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51c3d073-7877-4059-b1c8-e6f477af8436-tls-certs\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-home\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-dshm\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-tmp-dir\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84nss\" (UniqueName: \"kubernetes.io/projected/51c3d073-7877-4059-b1c8-e6f477af8436-kube-api-access-84nss\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419576 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419462    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-model-cache\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419682 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419640    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-home\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.419765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.419737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-tmp-dir\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.421791 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.421760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-dshm\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.421990 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.421974    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51c3d073-7877-4059-b1c8-e6f477af8436-tls-certs\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.427552 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.427528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84nss\" (UniqueName: \"kubernetes.io/projected/51c3d073-7877-4059-b1c8-e6f477af8436-kube-api-access-84nss\") pod \"test-model-8d5457-kserve-5dccf947d7-jmrtg\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") " pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.455654 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.455628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:06:46.464717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.464696    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:46.740243 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:46.740217    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"]
Sep 06 01:06:46.741425 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:06:46.741388    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51c3d073_7877_4059_b1c8_e6f477af8436.slice/crio-803e1540cd11016d7d11324aeae1ee3965eaa82ac31537d0bcaf1688acfe3e5a WatchSource:0}: Error finding container 803e1540cd11016d7d11324aeae1ee3965eaa82ac31537d0bcaf1688acfe3e5a: Status 404 returned error can't find the container with id 803e1540cd11016d7d11324aeae1ee3965eaa82ac31537d0bcaf1688acfe3e5a
Sep 06 01:06:47.242661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:47.242626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg" event={"ID":"51c3d073-7877-4059-b1c8-e6f477af8436","Type":"ContainerStarted","Data":"de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380"}
Sep 06 01:06:47.242661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:47.242665    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg" event={"ID":"51c3d073-7877-4059-b1c8-e6f477af8436","Type":"ContainerStarted","Data":"803e1540cd11016d7d11324aeae1ee3965eaa82ac31537d0bcaf1688acfe3e5a"}
Sep 06 01:06:47.258937 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:47.258875    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg" podStartSLOduration=1.25884413 podStartE2EDuration="1.25884413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:06:47.257726161 +0000 UTC m=+969.587085008" watchObservedRunningTime="2026-09-06 01:06:47.25884413 +0000 UTC m=+969.588202966"
Sep 06 01:06:53.218567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:53.218531    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:06:53.219433 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:53.219415    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:06:55.233036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:55.233004    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:06:55.233403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:55.233233    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:06:56.465467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:56.465418    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:56.466398 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:56.466371    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:56.489333 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:56.489306    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:06:57.331898 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:06:57.331867    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:07:01.199323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.199288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:01.347100 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.347070    2575 generic.go:356] "Generic (PLEG): container finished" podID="ed2b8040-3024-4378-88c8-41130ab222cd" containerID="fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7" exitCode=0
Sep 06 01:07:01.347265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.347105    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" event={"ID":"ed2b8040-3024-4378-88c8-41130ab222cd","Type":"ContainerDied","Data":"fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"}
Sep 06 01:07:01.347572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.347550    2575 scope.go:117] "RemoveContainer" containerID="fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"
Sep 06 01:07:01.348662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.348628    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:07:01.348743 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.348699    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:07:01.348743 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:01.348729    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:07:02.356954 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:02.356914    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" event={"ID":"ed2b8040-3024-4378-88c8-41130ab222cd","Type":"ContainerStarted","Data":"a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de"}
Sep 06 01:07:02.452960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:02.452927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:07:02.453417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:02.453338    2575 patch_prober.go:28] interesting pod/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 01:07:02.453589 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:02.453567    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 01:07:03.169837 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.169800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:03.453228 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.453195    2575 patch_prober.go:28] interesting pod/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 01:07:03.453778 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.453746    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 01:07:03.461536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.461501    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"]
Sep 06 01:07:03.476777 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.476736    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"]
Sep 06 01:07:03.476971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.476885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:03.480903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.480870    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:07:03.481090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.481066    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:07:03.594624 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.594574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaee620-2ba7-4cca-879a-8fa010c6a828-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk\" (UID: \"bdaee620-2ba7-4cca-879a-8fa010c6a828\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:03.696000 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.695963    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaee620-2ba7-4cca-879a-8fa010c6a828-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk\" (UID: \"bdaee620-2ba7-4cca-879a-8fa010c6a828\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:03.696603 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.696580    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaee620-2ba7-4cca-879a-8fa010c6a828-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk\" (UID: \"bdaee620-2ba7-4cca-879a-8fa010c6a828\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:03.792994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.792897    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:03.981388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:03.981341    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:04.065211 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:04.065120    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"]
Sep 06 01:07:04.078873 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:07:04.078824    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdaee620_2ba7_4cca_879a_8fa010c6a828.slice/crio-1edcc8ad0105772329b39d376709e25fa8f5d38bd2378a1237464d7997ecee25 WatchSource:0}: Error finding container 1edcc8ad0105772329b39d376709e25fa8f5d38bd2378a1237464d7997ecee25: Status 404 returned error can't find the container with id 1edcc8ad0105772329b39d376709e25fa8f5d38bd2378a1237464d7997ecee25
Sep 06 01:07:04.378911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:04.378782    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk" event={"ID":"bdaee620-2ba7-4cca-879a-8fa010c6a828","Type":"ContainerStarted","Data":"1edcc8ad0105772329b39d376709e25fa8f5d38bd2378a1237464d7997ecee25"}
Sep 06 01:07:04.453987 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:04.453480    2575 patch_prober.go:28] interesting pod/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 01:07:04.453987 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:04.453543    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 01:07:04.652297 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:04.652193    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:05.171308 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:05.171263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:05.386262 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:05.386168    2575 generic.go:356] "Generic (PLEG): container finished" podID="bdaee620-2ba7-4cca-879a-8fa010c6a828" containerID="5cc3df5edf5fe9a48e5151d8b9a6984e0d3a8fd680bb8998e4e1791f96df614d" exitCode=0
Sep 06 01:07:05.386262 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:05.386229    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk" event={"ID":"bdaee620-2ba7-4cca-879a-8fa010c6a828","Type":"ContainerDied","Data":"5cc3df5edf5fe9a48e5151d8b9a6984e0d3a8fd680bb8998e4e1791f96df614d"}
Sep 06 01:07:05.453262 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:05.453221    2575 patch_prober.go:28] interesting pod/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 01:07:05.453462 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:05.453285    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 01:07:06.460878 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:06.457247    2575 patch_prober.go:28] interesting pod/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 01:07:06.460878 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:06.457307    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 01:07:06.534126 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:06.534103    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:06.622434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:06.622402    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaee620-2ba7-4cca-879a-8fa010c6a828-maas-api-service-ca\") pod \"bdaee620-2ba7-4cca-879a-8fa010c6a828\" (UID: \"bdaee620-2ba7-4cca-879a-8fa010c6a828\") "
Sep 06 01:07:06.622736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:06.622712    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdaee620-2ba7-4cca-879a-8fa010c6a828-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bdaee620-2ba7-4cca-879a-8fa010c6a828" (UID: "bdaee620-2ba7-4cca-879a-8fa010c6a828"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:06.723229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:06.723199    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdaee620-2ba7-4cca-879a-8fa010c6a828-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:07.399081 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:07.399050    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"
Sep 06 01:07:07.399265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:07.399085    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk" event={"ID":"bdaee620-2ba7-4cca-879a-8fa010c6a828","Type":"ContainerDied","Data":"1edcc8ad0105772329b39d376709e25fa8f5d38bd2378a1237464d7997ecee25"}
Sep 06 01:07:07.399265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:07.399131    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1edcc8ad0105772329b39d376709e25fa8f5d38bd2378a1237464d7997ecee25"
Sep 06 01:07:07.453509 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:07.453469    2575 patch_prober.go:28] interesting pod/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 06 01:07:07.453665 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:07.453537    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 06 01:07:08.176807 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:08.176775    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:07:08.456860 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:08.456829    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:07:08.457790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:08.457772    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:07:10.034962 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.034930    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:10.658331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.658292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:10.703043 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.703007    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"]
Sep 06 01:07:10.703252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.703229    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk" podUID="b023a635-0eff-4b81-8844-64120111a728" containerName="payload-processing" containerID="cri-o://e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd" gracePeriod=30
Sep 06 01:07:10.713812 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.713786    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"]
Sep 06 01:07:10.714029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.714008    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl" podUID="977adebd-9ca3-4795-8d5a-7e54f64cc619" containerName="payload-pre-processing" containerID="cri-o://fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b" gracePeriod=30
Sep 06 01:07:10.751820 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.751787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:10.823781 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.823728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:10.898432 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.898402    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:10.898763 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.898748    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"]
Sep 06 01:07:10.902686 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.902661    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8a9d2066-5mfrk"]
Sep 06 01:07:10.977044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.977022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:10.980257 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.980238    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:07:10.983362 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:10.983344    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:07:11.041618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.041587    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"]
Sep 06 01:07:11.042098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.041803    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy" containerID="cri-o://a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de" gracePeriod=30
Sep 06 01:07:11.066210 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.066183    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc2jm\" (UniqueName: \"kubernetes.io/projected/977adebd-9ca3-4795-8d5a-7e54f64cc619-kube-api-access-cc2jm\") pod \"977adebd-9ca3-4795-8d5a-7e54f64cc619\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") "
Sep 06 01:07:11.066403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.066254    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977adebd-9ca3-4795-8d5a-7e54f64cc619-plugins-config-volume\") pod \"977adebd-9ca3-4795-8d5a-7e54f64cc619\" (UID: \"977adebd-9ca3-4795-8d5a-7e54f64cc619\") "
Sep 06 01:07:11.066403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.066307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jkpr\" (UniqueName: \"kubernetes.io/projected/b023a635-0eff-4b81-8844-64120111a728-kube-api-access-8jkpr\") pod \"b023a635-0eff-4b81-8844-64120111a728\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") "
Sep 06 01:07:11.066403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.066354    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b023a635-0eff-4b81-8844-64120111a728-plugins-config-volume\") pod \"b023a635-0eff-4b81-8844-64120111a728\" (UID: \"b023a635-0eff-4b81-8844-64120111a728\") "
Sep 06 01:07:11.066763 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.066724    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/977adebd-9ca3-4795-8d5a-7e54f64cc619-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "977adebd-9ca3-4795-8d5a-7e54f64cc619" (UID: "977adebd-9ca3-4795-8d5a-7e54f64cc619"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:11.066918 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.066770    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b023a635-0eff-4b81-8844-64120111a728-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b023a635-0eff-4b81-8844-64120111a728" (UID: "b023a635-0eff-4b81-8844-64120111a728"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:11.067206 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.067182    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:11.068499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.068476    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b023a635-0eff-4b81-8844-64120111a728-kube-api-access-8jkpr" (OuterVolumeSpecName: "kube-api-access-8jkpr") pod "b023a635-0eff-4b81-8844-64120111a728" (UID: "b023a635-0eff-4b81-8844-64120111a728"). InnerVolumeSpecName "kube-api-access-8jkpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:11.068897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.068876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/977adebd-9ca3-4795-8d5a-7e54f64cc619-kube-api-access-cc2jm" (OuterVolumeSpecName: "kube-api-access-cc2jm") pod "977adebd-9ca3-4795-8d5a-7e54f64cc619" (UID: "977adebd-9ca3-4795-8d5a-7e54f64cc619"). InnerVolumeSpecName "kube-api-access-cc2jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:11.167681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.167646    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc2jm\" (UniqueName: \"kubernetes.io/projected/977adebd-9ca3-4795-8d5a-7e54f64cc619-kube-api-access-cc2jm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:11.167681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.167681    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/977adebd-9ca3-4795-8d5a-7e54f64cc619-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:11.167966 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.167695    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jkpr\" (UniqueName: \"kubernetes.io/projected/b023a635-0eff-4b81-8844-64120111a728-kube-api-access-8jkpr\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:11.167966 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.167724    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b023a635-0eff-4b81-8844-64120111a728-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:11.422834 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.422747    2575 generic.go:356] "Generic (PLEG): container finished" podID="b023a635-0eff-4b81-8844-64120111a728" containerID="e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd" exitCode=0
Sep 06 01:07:11.422834 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.422795    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"
Sep 06 01:07:11.423080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.422831    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk" event={"ID":"b023a635-0eff-4b81-8844-64120111a728","Type":"ContainerDied","Data":"e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd"}
Sep 06 01:07:11.423080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.422898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk" event={"ID":"b023a635-0eff-4b81-8844-64120111a728","Type":"ContainerDied","Data":"e1bcbce321e344d43820c16b98830780e37b6919a41b64d4e094336b34536e04"}
Sep 06 01:07:11.423080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.422921    2575 scope.go:117] "RemoveContainer" containerID="e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd"
Sep 06 01:07:11.424535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.424452    2575 generic.go:356] "Generic (PLEG): container finished" podID="977adebd-9ca3-4795-8d5a-7e54f64cc619" containerID="fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b" exitCode=0
Sep 06 01:07:11.424535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.424496    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"
Sep 06 01:07:11.424535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.424531    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl" event={"ID":"977adebd-9ca3-4795-8d5a-7e54f64cc619","Type":"ContainerDied","Data":"fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b"}
Sep 06 01:07:11.424777 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.424573    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl" event={"ID":"977adebd-9ca3-4795-8d5a-7e54f64cc619","Type":"ContainerDied","Data":"58ff0ed03ccd5fd13c0d6da20dfe918586a42b37632fc2289531ecaa987533a0"}
Sep 06 01:07:11.432838 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.432822    2575 scope.go:117] "RemoveContainer" containerID="e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd"
Sep 06 01:07:11.433170 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:07:11.433152    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd\": container with ID starting with e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd not found: ID does not exist" containerID="e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd"
Sep 06 01:07:11.433248 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.433179    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd"} err="failed to get container status \"e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd\": rpc error: code = NotFound desc = could not find container \"e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd\": container with ID starting with e926224efba1fdbbf07ffaca66a911f63264507ed8ffa35ba9c852c0c77c46fd not found: ID does not exist"
Sep 06 01:07:11.433248 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.433195    2575 scope.go:117] "RemoveContainer" containerID="fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b"
Sep 06 01:07:11.441725 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.441708    2575 scope.go:117] "RemoveContainer" containerID="fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b"
Sep 06 01:07:11.442005 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:07:11.441986    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b\": container with ID starting with fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b not found: ID does not exist" containerID="fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b"
Sep 06 01:07:11.442051 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.442013    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b"} err="failed to get container status \"fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b\": rpc error: code = NotFound desc = could not find container \"fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b\": container with ID starting with fc5898e46db4ccfe0c97ced976988543d5e486cc3450774ab470fcd4d538ed8b not found: ID does not exist"
Sep 06 01:07:11.451436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.451417    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"]
Sep 06 01:07:11.455229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.455209    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a9d2066-5b48899474-bkkfl"]
Sep 06 01:07:11.465411 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.465389    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"]
Sep 06 01:07:11.469156 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.469138    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a9d2066-6cf669785-bwpzk"]
Sep 06 01:07:11.808572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.808540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:11.904960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:11.904921    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:12.002894 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:12.002845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:12.202212 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:12.202178    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="977adebd-9ca3-4795-8d5a-7e54f64cc619" path="/var/lib/kubelet/pods/977adebd-9ca3-4795-8d5a-7e54f64cc619/volumes"
Sep 06 01:07:12.202593 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:12.202579    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b023a635-0eff-4b81-8844-64120111a728" path="/var/lib/kubelet/pods/b023a635-0eff-4b81-8844-64120111a728/volumes"
Sep 06 01:07:12.202922 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:12.202908    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdaee620-2ba7-4cca-879a-8fa010c6a828" path="/var/lib/kubelet/pods/bdaee620-2ba7-4cca-879a-8fa010c6a828/volumes"
Sep 06 01:07:13.472069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:13.472039    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:16.294111 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.294089    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:07:16.416219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416145    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-certs\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416183    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2b8040-3024-4378-88c8-41130ab222cd-istiod-ca-cert\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416216    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-envoy\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416266    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-istio-token\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416297    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-credential-socket\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416377    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-socket\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416405    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-data\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416441    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2b8040-3024-4378-88c8-41130ab222cd-istio-podinfo\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416471    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v454\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-kube-api-access-5v454\") pod \"ed2b8040-3024-4378-88c8-41130ab222cd\" (UID: \"ed2b8040-3024-4378-88c8-41130ab222cd\") "
Sep 06 01:07:16.416490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416478    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:16.416843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416598    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2b8040-3024-4378-88c8-41130ab222cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:16.416843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416705    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:16.416843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416682    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:16.416843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416779    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:16.417250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416902    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.417250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416924    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.417250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416938    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2b8040-3024-4378-88c8-41130ab222cd-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.417250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.416951    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.418693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.418668    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:16.418693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.418682    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed2b8040-3024-4378-88c8-41130ab222cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:07:16.418693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.418680    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:16.418914 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.418784    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-kube-api-access-5v454" (OuterVolumeSpecName: "kube-api-access-5v454") pod "ed2b8040-3024-4378-88c8-41130ab222cd" (UID: "ed2b8040-3024-4378-88c8-41130ab222cd"). InnerVolumeSpecName "kube-api-access-5v454". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:16.455707 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.455676    2575 generic.go:356] "Generic (PLEG): container finished" podID="ed2b8040-3024-4378-88c8-41130ab222cd" containerID="a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de" exitCode=0
Sep 06 01:07:16.455880 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.455736    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"
Sep 06 01:07:16.455880 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.455752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" event={"ID":"ed2b8040-3024-4378-88c8-41130ab222cd","Type":"ContainerDied","Data":"a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de"}
Sep 06 01:07:16.455880 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.455803    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z" event={"ID":"ed2b8040-3024-4378-88c8-41130ab222cd","Type":"ContainerDied","Data":"a01460e572ab6daac7fa5872e15d2cb86d53b3bd9cda194b7a6eff6cd74842b3"}
Sep 06 01:07:16.455880 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.455825    2575 scope.go:117] "RemoveContainer" containerID="a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de"
Sep 06 01:07:16.465667 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.465650    2575 scope.go:117] "RemoveContainer" containerID="fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"
Sep 06 01:07:16.474339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.474324    2575 scope.go:117] "RemoveContainer" containerID="a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de"
Sep 06 01:07:16.474689 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:07:16.474655    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de\": container with ID starting with a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de not found: ID does not exist" containerID="a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de"
Sep 06 01:07:16.474898 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.474689    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de"} err="failed to get container status \"a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de\": rpc error: code = NotFound desc = could not find container \"a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de\": container with ID starting with a88d1eb9c4b58682d4def3298ce12dd97494386aed320d0a13ec6a5469d920de not found: ID does not exist"
Sep 06 01:07:16.474898 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.474739    2575 scope.go:117] "RemoveContainer" containerID="fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"
Sep 06 01:07:16.475126 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:07:16.475099    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7\": container with ID starting with fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7 not found: ID does not exist" containerID="fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"
Sep 06 01:07:16.475184 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.475129    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7"} err="failed to get container status \"fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7\": rpc error: code = NotFound desc = could not find container \"fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7\": container with ID starting with fc160fe572c0f477c52e7837daa8b7d21dead6fd710303e3d3ff6133e5ffc9a7 not found: ID does not exist"
Sep 06 01:07:16.477735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.477717    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"]
Sep 06 01:07:16.481992 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.481974    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8a9d2066-openshift-default-5b56d6d5b4-tgd5z"]
Sep 06 01:07:16.517642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.517619    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.517642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.517639    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2b8040-3024-4378-88c8-41130ab222cd-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.517779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.517648    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v454\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-kube-api-access-5v454\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.517779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.517657    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2b8040-3024-4378-88c8-41130ab222cd-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:16.517779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:16.517666    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2b8040-3024-4378-88c8-41130ab222cd-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:18.204542 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:18.204509    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" path="/var/lib/kubelet/pods/ed2b8040-3024-4378-88c8-41130ab222cd/volumes"
Sep 06 01:07:19.386961 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:19.386923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:19.500397 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:19.500360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:19.590064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:19.590032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:19.690097 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:19.690021    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:20.135904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:20.135827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:30.364202 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364166    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"]
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364516    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy"
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364527    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy"
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364535    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdaee620-2ba7-4cca-879a-8fa010c6a828" containerName="revoke-keys"
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364540    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdaee620-2ba7-4cca-879a-8fa010c6a828" containerName="revoke-keys"
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364554    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="977adebd-9ca3-4795-8d5a-7e54f64cc619" containerName="payload-pre-processing"
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364560    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="977adebd-9ca3-4795-8d5a-7e54f64cc619" containerName="payload-pre-processing"
Sep 06 01:07:30.364567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364567    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b023a635-0eff-4b81-8844-64120111a728" containerName="payload-processing"
Sep 06 01:07:30.364784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364573    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b023a635-0eff-4b81-8844-64120111a728" containerName="payload-processing"
Sep 06 01:07:30.364784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364632    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy"
Sep 06 01:07:30.364784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364639    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy"
Sep 06 01:07:30.364784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364647    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b023a635-0eff-4b81-8844-64120111a728" containerName="payload-processing"
Sep 06 01:07:30.364784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364653    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="977adebd-9ca3-4795-8d5a-7e54f64cc619" containerName="payload-pre-processing"
Sep 06 01:07:30.364784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.364658    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdaee620-2ba7-4cca-879a-8fa010c6a828" containerName="revoke-keys"
Sep 06 01:07:30.367314 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.367297    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.370654 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.370632    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-1fc789\"/\"openshift-service-ca.crt\""
Sep 06 01:07:30.370654 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.370648    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-1fc789\"/\"default-dockercfg-t2mhs\""
Sep 06 01:07:30.370832 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.370668    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-1fc789\"/\"kube-root-ca.crt\""
Sep 06 01:07:30.370832 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.370705    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-1fc789\"/\"explicit-model-1fc789-kserve-self-signed-certs\""
Sep 06 01:07:30.378737 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.378715    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"]
Sep 06 01:07:30.466118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.466084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:30.548423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.548392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-model-cache\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.548618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.548440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h499m\" (UniqueName: \"kubernetes.io/projected/0efc25a3-5f79-4989-9827-c83ab557e0a6-kube-api-access-h499m\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.548618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.548534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efc25a3-5f79-4989-9827-c83ab557e0a6-tls-certs\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.548618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.548571    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-tmp-dir\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.548618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.548597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-dshm\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.548767 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.548637    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-home\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.649582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.649502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-dshm\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.649582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.649541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-home\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.649811 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.649586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-model-cache\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.649811 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.649617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h499m\" (UniqueName: \"kubernetes.io/projected/0efc25a3-5f79-4989-9827-c83ab557e0a6-kube-api-access-h499m\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.649811 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.649673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efc25a3-5f79-4989-9827-c83ab557e0a6-tls-certs\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.649811 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.649718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-tmp-dir\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.650085 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.650052    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-home\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.650137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.650090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-model-cache\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.650137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.650111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-tmp-dir\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.652025 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.652002    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-dshm\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.652390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.652367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efc25a3-5f79-4989-9827-c83ab557e0a6-tls-certs\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.660393 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.660368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h499m\" (UniqueName: \"kubernetes.io/projected/0efc25a3-5f79-4989-9827-c83ab557e0a6-kube-api-access-h499m\") pod \"explicit-model-1fc789-kserve-868bdb4d5c-t4kpx\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") " pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.677230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.677212    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:30.953053 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:30.953027    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"]
Sep 06 01:07:30.954322 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:07:30.954294    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0efc25a3_5f79_4989_9827_c83ab557e0a6.slice/crio-36e1d62752b3623bf714097c99a27f8cbcb61d8e10c37c92294fbe34f821faad WatchSource:0}: Error finding container 36e1d62752b3623bf714097c99a27f8cbcb61d8e10c37c92294fbe34f821faad: Status 404 returned error can't find the container with id 36e1d62752b3623bf714097c99a27f8cbcb61d8e10c37c92294fbe34f821faad
Sep 06 01:07:31.539815 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:31.539780    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx" event={"ID":"0efc25a3-5f79-4989-9827-c83ab557e0a6","Type":"ContainerStarted","Data":"6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d"}
Sep 06 01:07:31.540209 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:31.539826    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx" event={"ID":"0efc25a3-5f79-4989-9827-c83ab557e0a6","Type":"ContainerStarted","Data":"36e1d62752b3623bf714097c99a27f8cbcb61d8e10c37c92294fbe34f821faad"}
Sep 06 01:07:31.560365 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:31.560321    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx" podStartSLOduration=1.560306349 podStartE2EDuration="1.560306349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:07:31.557222732 +0000 UTC m=+1013.886581579" watchObservedRunningTime="2026-09-06 01:07:31.560306349 +0000 UTC m=+1013.889665185"
Sep 06 01:07:32.824127 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.824092    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"]
Sep 06 01:07:32.844060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.844028    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:32.946810 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.946778    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"]
Sep 06 01:07:32.947162 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.947150    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy"
Sep 06 01:07:32.947213 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.947164    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2b8040-3024-4378-88c8-41130ab222cd" containerName="istio-proxy"
Sep 06 01:07:32.949098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.949083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:32.951487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.951464    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:07:32.951618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.951464    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:07:32.959011 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.958987    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"]
Sep 06 01:07:32.968789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:32.968752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6716c782-d396-401f-b178-237a895c93ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-1fc789-chjjz\" (UID: \"6716c782-d396-401f-b178-237a895c93ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:33.060064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:33.060024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:33.069210 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:33.069185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6716c782-d396-401f-b178-237a895c93ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-1fc789-chjjz\" (UID: \"6716c782-d396-401f-b178-237a895c93ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:33.069744 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:33.069726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6716c782-d396-401f-b178-237a895c93ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-1fc789-chjjz\" (UID: \"6716c782-d396-401f-b178-237a895c93ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:33.260591 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:33.260556    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:33.531914 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:33.531875    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"]
Sep 06 01:07:33.539765 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:07:33.539736    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6716c782_d396_401f_b178_237a895c93ae.slice/crio-233635a70b2f84d24747d3e71fc1c593a4ba300ca3d005ddae4f5713b8fb1d7a WatchSource:0}: Error finding container 233635a70b2f84d24747d3e71fc1c593a4ba300ca3d005ddae4f5713b8fb1d7a: Status 404 returned error can't find the container with id 233635a70b2f84d24747d3e71fc1c593a4ba300ca3d005ddae4f5713b8fb1d7a
Sep 06 01:07:33.551883 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:33.551842    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz" event={"ID":"6716c782-d396-401f-b178-237a895c93ae","Type":"ContainerStarted","Data":"233635a70b2f84d24747d3e71fc1c593a4ba300ca3d005ddae4f5713b8fb1d7a"}
Sep 06 01:07:34.559629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:34.559535    2575 generic.go:356] "Generic (PLEG): container finished" podID="6716c782-d396-401f-b178-237a895c93ae" containerID="13c10fbbd3e1cd23fd52c036d78b7342e0f299506bc2d9b1b56e1cfb97d5b2ed" exitCode=0
Sep 06 01:07:34.559629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:34.559581    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz" event={"ID":"6716c782-d396-401f-b178-237a895c93ae","Type":"ContainerDied","Data":"13c10fbbd3e1cd23fd52c036d78b7342e0f299506bc2d9b1b56e1cfb97d5b2ed"}
Sep 06 01:07:35.697573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:35.697550    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:35.792098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:35.792070    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6716c782-d396-401f-b178-237a895c93ae-maas-api-service-ca\") pod \"6716c782-d396-401f-b178-237a895c93ae\" (UID: \"6716c782-d396-401f-b178-237a895c93ae\") "
Sep 06 01:07:35.792453 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:35.792429    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6716c782-d396-401f-b178-237a895c93ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6716c782-d396-401f-b178-237a895c93ae" (UID: "6716c782-d396-401f-b178-237a895c93ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:35.892949 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:35.892878    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6716c782-d396-401f-b178-237a895c93ae-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:36.572577 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:36.572487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz" event={"ID":"6716c782-d396-401f-b178-237a895c93ae","Type":"ContainerDied","Data":"233635a70b2f84d24747d3e71fc1c593a4ba300ca3d005ddae4f5713b8fb1d7a"}
Sep 06 01:07:36.572577 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:36.572534    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="233635a70b2f84d24747d3e71fc1c593a4ba300ca3d005ddae4f5713b8fb1d7a"
Sep 06 01:07:36.572577 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:36.572500    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"
Sep 06 01:07:40.677496 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:40.677456    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:43.372390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.372356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:43.401509 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.401474    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"]
Sep 06 01:07:43.401737 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.401711    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2" podUID="ca6c1d59-ac15-45fd-9191-af063349df66" containerName="payload-processing" containerID="cri-o://af77973b2a57879c7c845b3fbfa7cb53284a294317090fff5b795dd4e6dc8673" gracePeriod=30
Sep 06 01:07:43.408967 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.408941    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"]
Sep 06 01:07:43.409167 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.409145    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd" podUID="64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" containerName="payload-pre-processing" containerID="cri-o://80be7dafc9c75a9d2674f9c7b55dda6092bf008b50d381a78f9d8c5b4b771783" gracePeriod=30
Sep 06 01:07:43.449652 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.449619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:43.514657 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.514628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:43.566269 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.566232    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"]
Sep 06 01:07:43.568588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.568566    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1fc789-chjjz"]
Sep 06 01:07:43.578410 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.578385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:43.617006 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.616981    2575 generic.go:356] "Generic (PLEG): container finished" podID="64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" containerID="80be7dafc9c75a9d2674f9c7b55dda6092bf008b50d381a78f9d8c5b4b771783" exitCode=0
Sep 06 01:07:43.617138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.617055    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd" event={"ID":"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d","Type":"ContainerDied","Data":"80be7dafc9c75a9d2674f9c7b55dda6092bf008b50d381a78f9d8c5b4b771783"}
Sep 06 01:07:43.618751 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.618727    2575 generic.go:356] "Generic (PLEG): container finished" podID="ca6c1d59-ac15-45fd-9191-af063349df66" containerID="af77973b2a57879c7c845b3fbfa7cb53284a294317090fff5b795dd4e6dc8673" exitCode=0
Sep 06 01:07:43.618893 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.618762    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2" event={"ID":"ca6c1d59-ac15-45fd-9191-af063349df66","Type":"ContainerDied","Data":"af77973b2a57879c7c845b3fbfa7cb53284a294317090fff5b795dd4e6dc8673"}
Sep 06 01:07:43.699191 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.699168    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:07:43.702036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.702016    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:07:43.760844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.760814    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlcmj\" (UniqueName: \"kubernetes.io/projected/ca6c1d59-ac15-45fd-9191-af063349df66-kube-api-access-dlcmj\") pod \"ca6c1d59-ac15-45fd-9191-af063349df66\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") "
Sep 06 01:07:43.761023 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.760878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-plugins-config-volume\") pod \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") "
Sep 06 01:07:43.761023 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.760951    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjp9w\" (UniqueName: \"kubernetes.io/projected/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-kube-api-access-vjp9w\") pod \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\" (UID: \"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d\") "
Sep 06 01:07:43.761023 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.760981    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c1d59-ac15-45fd-9191-af063349df66-plugins-config-volume\") pod \"ca6c1d59-ac15-45fd-9191-af063349df66\" (UID: \"ca6c1d59-ac15-45fd-9191-af063349df66\") "
Sep 06 01:07:43.761285 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.761260    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" (UID: "64ae230f-a9d9-4c8a-84f8-eddcbc017a9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:43.761421 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.761399    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6c1d59-ac15-45fd-9191-af063349df66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca6c1d59-ac15-45fd-9191-af063349df66" (UID: "ca6c1d59-ac15-45fd-9191-af063349df66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:07:43.763195 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.763171    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-kube-api-access-vjp9w" (OuterVolumeSpecName: "kube-api-access-vjp9w") pod "64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" (UID: "64ae230f-a9d9-4c8a-84f8-eddcbc017a9d"). InnerVolumeSpecName "kube-api-access-vjp9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:43.763195 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.763182    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6c1d59-ac15-45fd-9191-af063349df66-kube-api-access-dlcmj" (OuterVolumeSpecName: "kube-api-access-dlcmj") pod "ca6c1d59-ac15-45fd-9191-af063349df66" (UID: "ca6c1d59-ac15-45fd-9191-af063349df66"). InnerVolumeSpecName "kube-api-access-dlcmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:43.862464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.862433    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjp9w\" (UniqueName: \"kubernetes.io/projected/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-kube-api-access-vjp9w\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:43.862464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.862459    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c1d59-ac15-45fd-9191-af063349df66-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:43.862464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.862468    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlcmj\" (UniqueName: \"kubernetes.io/projected/ca6c1d59-ac15-45fd-9191-af063349df66-kube-api-access-dlcmj\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:43.862644 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.862477    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:43.917273 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:43.917241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:44.201601 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.201567    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6716c782-d396-401f-b178-237a895c93ae" path="/var/lib/kubelet/pods/6716c782-d396-401f-b178-237a895c93ae/volumes"
Sep 06 01:07:44.626120 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.626045    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"
Sep 06 01:07:44.626120 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.626096    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd" event={"ID":"64ae230f-a9d9-4c8a-84f8-eddcbc017a9d","Type":"ContainerDied","Data":"6a4651381b1ead003a4567013535bcbd8e374417d74e6b7a256199ab586e7162"}
Sep 06 01:07:44.626655 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.626157    2575 scope.go:117] "RemoveContainer" containerID="80be7dafc9c75a9d2674f9c7b55dda6092bf008b50d381a78f9d8c5b4b771783"
Sep 06 01:07:44.627535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.627520    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"
Sep 06 01:07:44.627624 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.627516    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2" event={"ID":"ca6c1d59-ac15-45fd-9191-af063349df66","Type":"ContainerDied","Data":"31095b414ca5ddda5d284716c43c4a5608a79c318348b10ca41bf55f4a46692c"}
Sep 06 01:07:44.635583 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.635567    2575 scope.go:117] "RemoveContainer" containerID="af77973b2a57879c7c845b3fbfa7cb53284a294317090fff5b795dd4e6dc8673"
Sep 06 01:07:44.645564 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.645547    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"]
Sep 06 01:07:44.648536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.648517    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1fc789-8695c978b9-zkxt2"]
Sep 06 01:07:44.659751 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.659728    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"]
Sep 06 01:07:44.664917 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:44.664886    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1fc789-7bbbb95ff4-mcpmd"]
Sep 06 01:07:46.032846 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:46.032800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:46.202476 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:46.202443    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" path="/var/lib/kubelet/pods/64ae230f-a9d9-4c8a-84f8-eddcbc017a9d/volumes"
Sep 06 01:07:46.202808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:46.202796    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6c1d59-ac15-45fd-9191-af063349df66" path="/var/lib/kubelet/pods/ca6c1d59-ac15-45fd-9191-af063349df66/volumes"
Sep 06 01:07:48.572562 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.572528    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx" podUID="0efc25a3-5f79-4989-9827-c83ab557e0a6" containerName="main" containerID="cri-o://6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d" gracePeriod=45
Sep 06 01:07:48.824027 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.823970    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:48.905777 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.905753    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efc25a3-5f79-4989-9827-c83ab557e0a6-tls-certs\") pod \"0efc25a3-5f79-4989-9827-c83ab557e0a6\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") "
Sep 06 01:07:48.905946 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.905820    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-dshm\") pod \"0efc25a3-5f79-4989-9827-c83ab557e0a6\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") "
Sep 06 01:07:48.905946 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.905841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-home\") pod \"0efc25a3-5f79-4989-9827-c83ab557e0a6\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") "
Sep 06 01:07:48.905946 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.905895    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-model-cache\") pod \"0efc25a3-5f79-4989-9827-c83ab557e0a6\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") "
Sep 06 01:07:48.906077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.905962    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-tmp-dir\") pod \"0efc25a3-5f79-4989-9827-c83ab557e0a6\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") "
Sep 06 01:07:48.906077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906031    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h499m\" (UniqueName: \"kubernetes.io/projected/0efc25a3-5f79-4989-9827-c83ab557e0a6-kube-api-access-h499m\") pod \"0efc25a3-5f79-4989-9827-c83ab557e0a6\" (UID: \"0efc25a3-5f79-4989-9827-c83ab557e0a6\") "
Sep 06 01:07:48.906219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906144    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-home" (OuterVolumeSpecName: "home") pod "0efc25a3-5f79-4989-9827-c83ab557e0a6" (UID: "0efc25a3-5f79-4989-9827-c83ab557e0a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:48.906285 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906210    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "0efc25a3-5f79-4989-9827-c83ab557e0a6" (UID: "0efc25a3-5f79-4989-9827-c83ab557e0a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:48.906344 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906289    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0efc25a3-5f79-4989-9827-c83ab557e0a6" (UID: "0efc25a3-5f79-4989-9827-c83ab557e0a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:48.906422 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906408    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:48.906481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906428    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:48.906481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.906464    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:48.908188 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.908163    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0efc25a3-5f79-4989-9827-c83ab557e0a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0efc25a3-5f79-4989-9827-c83ab557e0a6" (UID: "0efc25a3-5f79-4989-9827-c83ab557e0a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:07:48.908188 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.908170    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-dshm" (OuterVolumeSpecName: "dshm") pod "0efc25a3-5f79-4989-9827-c83ab557e0a6" (UID: "0efc25a3-5f79-4989-9827-c83ab557e0a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:07:48.908291 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.908246    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efc25a3-5f79-4989-9827-c83ab557e0a6-kube-api-access-h499m" (OuterVolumeSpecName: "kube-api-access-h499m") pod "0efc25a3-5f79-4989-9827-c83ab557e0a6" (UID: "0efc25a3-5f79-4989-9827-c83ab557e0a6"). InnerVolumeSpecName "kube-api-access-h499m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:07:48.995227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:48.995197    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:49.007153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.007127    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efc25a3-5f79-4989-9827-c83ab557e0a6-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:49.007153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.007148    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efc25a3-5f79-4989-9827-c83ab557e0a6-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:49.007299 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.007157    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h499m\" (UniqueName: \"kubernetes.io/projected/0efc25a3-5f79-4989-9827-c83ab557e0a6-kube-api-access-h499m\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:07:49.659611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.659575    2575 generic.go:356] "Generic (PLEG): container finished" podID="0efc25a3-5f79-4989-9827-c83ab557e0a6" containerID="6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d" exitCode=0
Sep 06 01:07:49.660131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.659632    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"
Sep 06 01:07:49.660131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.659641    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx" event={"ID":"0efc25a3-5f79-4989-9827-c83ab557e0a6","Type":"ContainerDied","Data":"6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d"}
Sep 06 01:07:49.660131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.659701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx" event={"ID":"0efc25a3-5f79-4989-9827-c83ab557e0a6","Type":"ContainerDied","Data":"36e1d62752b3623bf714097c99a27f8cbcb61d8e10c37c92294fbe34f821faad"}
Sep 06 01:07:49.660131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.659728    2575 scope.go:117] "RemoveContainer" containerID="6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d"
Sep 06 01:07:49.671554 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.671537    2575 scope.go:117] "RemoveContainer" containerID="6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d"
Sep 06 01:07:49.671820 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:07:49.671800    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d\": container with ID starting with 6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d not found: ID does not exist" containerID="6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d"
Sep 06 01:07:49.671903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.671828    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d"} err="failed to get container status \"6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d\": rpc error: code = NotFound desc = could not find container \"6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d\": container with ID starting with 6d9b82f0547fd7fb4057f12c22a7da23c2465a53f8ade41e856cb8511a78a29d not found: ID does not exist"
Sep 06 01:07:49.684002 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.683978    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"]
Sep 06 01:07:49.691104 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:49.691085    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-1fc789/explicit-model-1fc789-kserve-868bdb4d5c-t4kpx"]
Sep 06 01:07:50.201723 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:50.201690    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0efc25a3-5f79-4989-9827-c83ab557e0a6" path="/var/lib/kubelet/pods/0efc25a3-5f79-4989-9827-c83ab557e0a6/volumes"
Sep 06 01:07:50.780912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:50.780879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:50.845695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:50.845664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:51.092191 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.092115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:51.518127 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.518098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:51.814976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.814904    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"]
Sep 06 01:07:51.815298 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815262    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6c1d59-ac15-45fd-9191-af063349df66" containerName="payload-processing"
Sep 06 01:07:51.815298 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815274    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6c1d59-ac15-45fd-9191-af063349df66" containerName="payload-processing"
Sep 06 01:07:51.815298 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815282    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0efc25a3-5f79-4989-9827-c83ab557e0a6" containerName="main"
Sep 06 01:07:51.815298 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815288    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0efc25a3-5f79-4989-9827-c83ab557e0a6" containerName="main"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815317    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6716c782-d396-401f-b178-237a895c93ae" containerName="revoke-keys"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815323    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6716c782-d396-401f-b178-237a895c93ae" containerName="revoke-keys"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815334    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" containerName="payload-pre-processing"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815340    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" containerName="payload-pre-processing"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815414    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6716c782-d396-401f-b178-237a895c93ae" containerName="revoke-keys"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815423    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6c1d59-ac15-45fd-9191-af063349df66" containerName="payload-processing"
Sep 06 01:07:51.815430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815429    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ae230f-a9d9-4c8a-84f8-eddcbc017a9d" containerName="payload-pre-processing"
Sep 06 01:07:51.815627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.815434    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0efc25a3-5f79-4989-9827-c83ab557e0a6" containerName="main"
Sep 06 01:07:51.820050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.820030    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.822639 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.822619    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-fb4286-openshift-default-dockercfg-mqrbn\""
Sep 06 01:07:51.829242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.829212    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"]
Sep 06 01:07:51.849942 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.849914    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:51.933622 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-socket\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a579fc-9182-4549-80f6-e93d78106f13-istio-podinfo\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933668    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-data\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts7l5\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-kube-api-access-ts7l5\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933744    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a579fc-9182-4549-80f6-e93d78106f13-istiod-ca-cert\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933972 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933789    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-credential-socket\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933972 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-certs\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933972 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933843    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-envoy\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:51.933972 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:51.933898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-istio-token\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.034905 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.034874    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a579fc-9182-4549-80f6-e93d78106f13-istio-podinfo\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.034911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-data\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.034932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7l5\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-kube-api-access-ts7l5\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.034954    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a579fc-9182-4549-80f6-e93d78106f13-istiod-ca-cert\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.034980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-credential-socket\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035008    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-certs\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035286 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-envoy\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035286 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-istio-token\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035286 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-socket\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-data\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-certs\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035560 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-credential-socket\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035574    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-socket\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.035739 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.035721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a579fc-9182-4549-80f6-e93d78106f13-istiod-ca-cert\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.038048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.038024    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-envoy\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.038314 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.038297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a579fc-9182-4549-80f6-e93d78106f13-istio-podinfo\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.042876 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.042830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7l5\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-kube-api-access-ts7l5\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.043077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.043060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-istio-token\") pod \"e2e-noten-fb4286-openshift-default-856c456576-z5dqr\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") " pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.131060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.130992    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:52.173717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.173689    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:52.232919 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.232882    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:52.341801 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.341769    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:52.409804 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.409778    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"]
Sep 06 01:07:52.411078 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:07:52.411052    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a579fc_9182_4549_80f6_e93d78106f13.slice/crio-3a0339b727b1a74234bde16bb2c8d4ea59f50a3c5f40983f971dd8cab654ee15 WatchSource:0}: Error finding container 3a0339b727b1a74234bde16bb2c8d4ea59f50a3c5f40983f971dd8cab654ee15: Status 404 returned error can't find the container with id 3a0339b727b1a74234bde16bb2c8d4ea59f50a3c5f40983f971dd8cab654ee15
Sep 06 01:07:52.411819 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.411702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:52.413300 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.413276    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:07:52.413365 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.413339    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:07:52.413407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.413379    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:07:52.496363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.496338    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:52.597041 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.597001    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:52.680069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.679978    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr" event={"ID":"75a579fc-9182-4549-80f6-e93d78106f13","Type":"ContainerStarted","Data":"fd0fa6d965fd5c2ba900c41811afe88571f2288753de6e1b8e7bdd7cc68bcf98"}
Sep 06 01:07:52.680069 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.680033    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr" event={"ID":"75a579fc-9182-4549-80f6-e93d78106f13","Type":"ContainerStarted","Data":"3a0339b727b1a74234bde16bb2c8d4ea59f50a3c5f40983f971dd8cab654ee15"}
Sep 06 01:07:52.700270 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:52.700214    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr" podStartSLOduration=1.70020084 podStartE2EDuration="1.70020084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:07:52.697712435 +0000 UTC m=+1035.027071292" watchObservedRunningTime="2026-09-06 01:07:52.70020084 +0000 UTC m=+1035.029559675"
Sep 06 01:07:53.043620 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.043586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:53.125523 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.125493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:53.131831 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.131812    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:53.136685 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.136660    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:53.225617 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.225575    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:07:53.685052 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.685026    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:07:53.685992 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:07:53.685972    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:08:08.429400 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:08.429360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:14.470765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:14.470723    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:14.582145 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:14.582097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:14.685904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:14.685849    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:25.194180 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:25.194142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:26.004517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:26.004483    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:32.129115 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:32.129036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:32.195980 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:32.195952    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:32.281250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:32.281215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:32.380535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:32.380459    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:32.505712 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:32.505676    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:33.135549 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:33.135513    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:33.233421 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:33.233385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:33.336413 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:33.336380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:34.843916 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:34.843876    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:40.959987 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:40.959941    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:41.029707 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:41.029667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:41.654568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:41.654530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:42.257876 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.257818    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"]
Sep 06 01:08:42.262497 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.262480    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.266604 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.266561    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-fb4286\"/\"kube-root-ca.crt\""
Sep 06 01:08:42.266846 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.266627    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-fb4286\"/\"default-dockercfg-gcrqb\""
Sep 06 01:08:42.266972 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.266830    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-fb4286\"/\"noten-model-fb4286-kserve-self-signed-certs\""
Sep 06 01:08:42.267068 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.267050    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-fb4286\"/\"openshift-service-ca.crt\""
Sep 06 01:08:42.270196 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.269877    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"]
Sep 06 01:08:42.289052 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.289025    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:42.389559 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.389529    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-model-cache\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.389721 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.389597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-dshm\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.389721 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.389622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tmp-dir\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.389721 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.389643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xbm5\" (UniqueName: \"kubernetes.io/projected/bc72d384-44d6-4ddf-bfdb-007f083d21eb-kube-api-access-8xbm5\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.389721 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.389663    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tls-certs\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.389721 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.389720    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-home\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.490719 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.490680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-dshm\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.490952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.490723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tmp-dir\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.490952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.490756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xbm5\" (UniqueName: \"kubernetes.io/projected/bc72d384-44d6-4ddf-bfdb-007f083d21eb-kube-api-access-8xbm5\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.490952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.490779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tls-certs\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.490952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.490822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-home\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.490952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.490881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-model-cache\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.491264 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.491242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tmp-dir\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.491333 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.491301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-home\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.491333 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.491317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-model-cache\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.493238 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.493220    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-dshm\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.493482 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.493465    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tls-certs\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.500074 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.500047    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xbm5\" (UniqueName: \"kubernetes.io/projected/bc72d384-44d6-4ddf-bfdb-007f083d21eb-kube-api-access-8xbm5\") pod \"noten-model-fb4286-kserve-5fb5fcc955-g7sht\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") " pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.572970 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.572906    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:42.845565 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.845541    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"]
Sep 06 01:08:42.847172 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:08:42.847137    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc72d384_44d6_4ddf_bfdb_007f083d21eb.slice/crio-69bdb443eecd30fffb9c2cd2cb4bc902007e73029077f211d58adbd59f5374d4 WatchSource:0}: Error finding container 69bdb443eecd30fffb9c2cd2cb4bc902007e73029077f211d58adbd59f5374d4: Status 404 returned error can't find the container with id 69bdb443eecd30fffb9c2cd2cb4bc902007e73029077f211d58adbd59f5374d4
Sep 06 01:08:42.967680 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.967642    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht" event={"ID":"bc72d384-44d6-4ddf-bfdb-007f083d21eb","Type":"ContainerStarted","Data":"17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f"}
Sep 06 01:08:42.967680 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.967686    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht" event={"ID":"bc72d384-44d6-4ddf-bfdb-007f083d21eb","Type":"ContainerStarted","Data":"69bdb443eecd30fffb9c2cd2cb4bc902007e73029077f211d58adbd59f5374d4"}
Sep 06 01:08:42.988379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:42.988330    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht" podStartSLOduration=0.988314728 podStartE2EDuration="988.314728ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:08:42.985225209 +0000 UTC m=+1085.314584046" watchObservedRunningTime="2026-09-06 01:08:42.988314728 +0000 UTC m=+1085.317673598"
Sep 06 01:08:44.257358 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:44.257326    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"]
Sep 06 01:08:44.278322 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:44.278295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:45.258059 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.258021    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"]
Sep 06 01:08:45.276545 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.276502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:45.387270 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.387231    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"]
Sep 06 01:08:45.390667 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.390650    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:45.393137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.393115    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:08:45.393266 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.393178    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:08:45.399481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.399450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"]
Sep 06 01:08:45.520059 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.519976    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5f6a1e-ecf3-4bed-830c-4511704bf100-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd\" (UID: \"7b5f6a1e-ecf3-4bed-830c-4511704bf100\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:45.619979 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.619946    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:45.621158 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.621138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5f6a1e-ecf3-4bed-830c-4511704bf100-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd\" (UID: \"7b5f6a1e-ecf3-4bed-830c-4511704bf100\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:45.621740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.621724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5f6a1e-ecf3-4bed-830c-4511704bf100-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd\" (UID: \"7b5f6a1e-ecf3-4bed-830c-4511704bf100\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:45.702919 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.702892    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:45.977165 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:45.977136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"]
Sep 06 01:08:45.978762 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:08:45.978735    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5f6a1e_ecf3_4bed_830c_4511704bf100.slice/crio-f903dcc90bffeb6545cd7c5fb539341350a7412dda8e618120a04789a5f72d11 WatchSource:0}: Error finding container f903dcc90bffeb6545cd7c5fb539341350a7412dda8e618120a04789a5f72d11: Status 404 returned error can't find the container with id f903dcc90bffeb6545cd7c5fb539341350a7412dda8e618120a04789a5f72d11
Sep 06 01:08:46.992487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:46.992452    2575 generic.go:356] "Generic (PLEG): container finished" podID="7b5f6a1e-ecf3-4bed-830c-4511704bf100" containerID="68ec4db4c9d4bdc58c32474dcb0b96f0d8db1f4cade0fec64ae472ac4c4b0322" exitCode=0
Sep 06 01:08:46.992876 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:46.992505    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd" event={"ID":"7b5f6a1e-ecf3-4bed-830c-4511704bf100","Type":"ContainerDied","Data":"68ec4db4c9d4bdc58c32474dcb0b96f0d8db1f4cade0fec64ae472ac4c4b0322"}
Sep 06 01:08:46.992876 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:46.992547    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd" event={"ID":"7b5f6a1e-ecf3-4bed-830c-4511704bf100","Type":"ContainerStarted","Data":"f903dcc90bffeb6545cd7c5fb539341350a7412dda8e618120a04789a5f72d11"}
Sep 06 01:08:47.699300 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:47.699257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:47.799384 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:47.799358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:47.943926 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:47.943894    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:48.127202 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.127176    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:48.227627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.227594    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"]
Sep 06 01:08:48.228219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.228199    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b5f6a1e-ecf3-4bed-830c-4511704bf100" containerName="revoke-keys"
Sep 06 01:08:48.228306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.228222    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b5f6a1e-ecf3-4bed-830c-4511704bf100" containerName="revoke-keys"
Sep 06 01:08:48.228374 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.228356    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b5f6a1e-ecf3-4bed-830c-4511704bf100" containerName="revoke-keys"
Sep 06 01:08:48.231139 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.231124    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.233436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.233417    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-40d55b26-openshift-default-dockercfg-j6wf4\""
Sep 06 01:08:48.242487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.242466    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"]
Sep 06 01:08:48.245998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.245974    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5f6a1e-ecf3-4bed-830c-4511704bf100-maas-api-service-ca\") pod \"7b5f6a1e-ecf3-4bed-830c-4511704bf100\" (UID: \"7b5f6a1e-ecf3-4bed-830c-4511704bf100\") "
Sep 06 01:08:48.246321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.246293    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b5f6a1e-ecf3-4bed-830c-4511704bf100-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b5f6a1e-ecf3-4bed-830c-4511704bf100" (UID: "7b5f6a1e-ecf3-4bed-830c-4511704bf100"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:08:48.246913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.246894    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:48.346830 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.346741    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-istio-token\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.346830 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.346788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-socket\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.346830 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.346819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-data\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.346870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cpdr\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-kube-api-access-6cpdr\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.346900    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad36fd-f812-462c-809f-95ca8423159b-istiod-ca-cert\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.346945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-credential-socket\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.347021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad36fd-f812-462c-809f-95ca8423159b-istio-podinfo\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347181 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.347080    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-certs\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347181 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.347108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-envoy\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.347248 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.347179    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b5f6a1e-ecf3-4bed-830c-4511704bf100-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:08:48.422888 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.422844    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:48.448224 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448182    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-envoy\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.448421 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-istio-token\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.448421 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-socket\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.448549 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-data\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.448825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-socket\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.448825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cpdr\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-kube-api-access-6cpdr\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.448825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad36fd-f812-462c-809f-95ca8423159b-istiod-ca-cert\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-credential-socket\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad36fd-f812-462c-809f-95ca8423159b-istio-podinfo\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.448951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-certs\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.449019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-data\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.449217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-credential-socket\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.449274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-certs\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.449681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.449663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad36fd-f812-462c-809f-95ca8423159b-istiod-ca-cert\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.451260 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.451229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-envoy\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.451680 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.451646    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad36fd-f812-462c-809f-95ca8423159b-istio-podinfo\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.458446 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.458423    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-istio-token\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.458986 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.458951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cpdr\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-kube-api-access-6cpdr\") pod \"e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") " pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.541976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.541941    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:48.818275 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.818251    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"]
Sep 06 01:08:48.819391 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:08:48.819364    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaad36fd_f812_462c_809f_95ca8423159b.slice/crio-24838905d46e37af1e39152eef1168d99eb53b5f51c0f847f74130f733de7e73 WatchSource:0}: Error finding container 24838905d46e37af1e39152eef1168d99eb53b5f51c0f847f74130f733de7e73: Status 404 returned error can't find the container with id 24838905d46e37af1e39152eef1168d99eb53b5f51c0f847f74130f733de7e73
Sep 06 01:08:48.821662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.821632    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:08:48.821713 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.821706    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:08:48.821754 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:48.821734    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:08:49.008743 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.008703    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" event={"ID":"aaad36fd-f812-462c-809f-95ca8423159b","Type":"ContainerStarted","Data":"3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"}
Sep 06 01:08:49.008974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.008766    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" event={"ID":"aaad36fd-f812-462c-809f-95ca8423159b","Type":"ContainerStarted","Data":"24838905d46e37af1e39152eef1168d99eb53b5f51c0f847f74130f733de7e73"}
Sep 06 01:08:49.010309 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.010278    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd" event={"ID":"7b5f6a1e-ecf3-4bed-830c-4511704bf100","Type":"ContainerDied","Data":"f903dcc90bffeb6545cd7c5fb539341350a7412dda8e618120a04789a5f72d11"}
Sep 06 01:08:49.010445 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.010348    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f903dcc90bffeb6545cd7c5fb539341350a7412dda8e618120a04789a5f72d11"
Sep 06 01:08:49.010445 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.010293    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"
Sep 06 01:08:49.032115 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.032026    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podStartSLOduration=1.032007883 podStartE2EDuration="1.032007883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:08:49.030891936 +0000 UTC m=+1091.360250792" watchObservedRunningTime="2026-09-06 01:08:49.032007883 +0000 UTC m=+1091.361366723"
Sep 06 01:08:49.542700 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.542665    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:49.547922 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:49.547898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:50.016284 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:50.016255    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:50.017379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:50.017359    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:08:51.225245 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:51.225215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:51.351543 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:51.351515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:52.388123 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.388073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:52.455772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.455736    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:52.573887 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.573836    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:52.574076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.573903    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:52.586348 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.586325    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:52.645950 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.645879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:52.880271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.880241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:52.980835 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:52.980803    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:53.018405 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.018376    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"]
Sep 06 01:08:53.022362 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.022338    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.025108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.025074    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-40d55b26\""
Sep 06 01:08:53.025249 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.025074    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-40d55b26-dockercfg-ljbhp\""
Sep 06 01:08:53.041994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.041967    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"]
Sep 06 01:08:53.046048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.046029    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:08:53.093135 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.093106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56qh\" (UniqueName: \"kubernetes.io/projected/1b535f3b-443e-49e2-9e54-81828675a413-kube-api-access-h56qh\") pod \"payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") " pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.093268 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.093142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b535f3b-443e-49e2-9e54-81828675a413-plugins-config-volume\") pod \"payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") " pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.194286 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.194253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b535f3b-443e-49e2-9e54-81828675a413-plugins-config-volume\") pod \"payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") " pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.194477 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.194446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56qh\" (UniqueName: \"kubernetes.io/projected/1b535f3b-443e-49e2-9e54-81828675a413-kube-api-access-h56qh\") pod \"payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") " pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.195107 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.195086    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b535f3b-443e-49e2-9e54-81828675a413-plugins-config-volume\") pod \"payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") " pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.203530 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.203500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56qh\" (UniqueName: \"kubernetes.io/projected/1b535f3b-443e-49e2-9e54-81828675a413-kube-api-access-h56qh\") pod \"payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") " pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.334989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.334904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:53.610481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:53.610456    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"]
Sep 06 01:08:53.611941 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:08:53.611910    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b535f3b_443e_49e2_9e54_81828675a413.slice/crio-a2e5d2395e2814ea53d08184308218e6e4338dff2ef7b9f5ed24e3f8bca046ef WatchSource:0}: Error finding container a2e5d2395e2814ea53d08184308218e6e4338dff2ef7b9f5ed24e3f8bca046ef: Status 404 returned error can't find the container with id a2e5d2395e2814ea53d08184308218e6e4338dff2ef7b9f5ed24e3f8bca046ef
Sep 06 01:08:54.040753 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:54.040710    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9" event={"ID":"1b535f3b-443e-49e2-9e54-81828675a413","Type":"ContainerStarted","Data":"deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2"}
Sep 06 01:08:54.040969 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:54.040891    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:08:54.040969 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:54.040944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9" event={"ID":"1b535f3b-443e-49e2-9e54-81828675a413","Type":"ContainerStarted","Data":"a2e5d2395e2814ea53d08184308218e6e4338dff2ef7b9f5ed24e3f8bca046ef"}
Sep 06 01:08:54.058356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:54.058311    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9" podStartSLOduration=2.058297321 podStartE2EDuration="2.058297321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:08:54.058037482 +0000 UTC m=+1096.387396329" watchObservedRunningTime="2026-09-06 01:08:54.058297321 +0000 UTC m=+1096.387656157"
Sep 06 01:08:55.520076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:55.520034    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:56.200443 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.200414    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:56.301383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.301345    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:56.388511 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.388467    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"]
Sep 06 01:08:56.390685 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.390659    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:56.391471 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.391452    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-701b5b-29mgd"]
Sep 06 01:08:56.446373 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.446343    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:56.773483 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:56.773451    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:08:58.202898 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:58.202846    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b5f6a1e-ecf3-4bed-830c-4511704bf100" path="/var/lib/kubelet/pods/7b5f6a1e-ecf3-4bed-830c-4511704bf100/volumes"
Sep 06 01:08:59.277979 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.277932    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27" podUID="ca30e9ec-7602-4745-8fd9-1fba1e55129a" containerName="main" containerID="cri-o://fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d" gracePeriod=45
Sep 06 01:08:59.531218 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.531162    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:08:59.655602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.655570    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-dshm\") pod \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") "
Sep 06 01:08:59.655762 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.655639    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz9cv\" (UniqueName: \"kubernetes.io/projected/ca30e9ec-7602-4745-8fd9-1fba1e55129a-kube-api-access-cz9cv\") pod \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") "
Sep 06 01:08:59.655762 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.655686    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tls-certs\") pod \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") "
Sep 06 01:08:59.655762 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.655739    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-home\") pod \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") "
Sep 06 01:08:59.655952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.655765    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tmp-dir\") pod \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") "
Sep 06 01:08:59.655952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.655793    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-model-cache\") pod \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\" (UID: \"ca30e9ec-7602-4745-8fd9-1fba1e55129a\") "
Sep 06 01:08:59.656059 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.656007    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-home" (OuterVolumeSpecName: "home") pod "ca30e9ec-7602-4745-8fd9-1fba1e55129a" (UID: "ca30e9ec-7602-4745-8fd9-1fba1e55129a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:08:59.656113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.656063    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca30e9ec-7602-4745-8fd9-1fba1e55129a" (UID: "ca30e9ec-7602-4745-8fd9-1fba1e55129a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:08:59.656196 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.656173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca30e9ec-7602-4745-8fd9-1fba1e55129a" (UID: "ca30e9ec-7602-4745-8fd9-1fba1e55129a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:08:59.658078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.658051    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca30e9ec-7602-4745-8fd9-1fba1e55129a-kube-api-access-cz9cv" (OuterVolumeSpecName: "kube-api-access-cz9cv") pod "ca30e9ec-7602-4745-8fd9-1fba1e55129a" (UID: "ca30e9ec-7602-4745-8fd9-1fba1e55129a"). InnerVolumeSpecName "kube-api-access-cz9cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:08:59.658078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.658074    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-dshm" (OuterVolumeSpecName: "dshm") pod "ca30e9ec-7602-4745-8fd9-1fba1e55129a" (UID: "ca30e9ec-7602-4745-8fd9-1fba1e55129a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:08:59.658078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.658078    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca30e9ec-7602-4745-8fd9-1fba1e55129a" (UID: "ca30e9ec-7602-4745-8fd9-1fba1e55129a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:08:59.757017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.756996    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz9cv\" (UniqueName: \"kubernetes.io/projected/ca30e9ec-7602-4745-8fd9-1fba1e55129a-kube-api-access-cz9cv\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:08:59.757017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.757019    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:08:59.757154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.757028    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:08:59.757154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.757037    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:08:59.757154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.757046    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:08:59.757154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:08:59.757054    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca30e9ec-7602-4745-8fd9-1fba1e55129a-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.084980 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.084942    2575 generic.go:356] "Generic (PLEG): container finished" podID="ca30e9ec-7602-4745-8fd9-1fba1e55129a" containerID="fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d" exitCode=0
Sep 06 01:09:00.084980 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.084983    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"
Sep 06 01:09:00.085192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.085020    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27" event={"ID":"ca30e9ec-7602-4745-8fd9-1fba1e55129a","Type":"ContainerDied","Data":"fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d"}
Sep 06 01:09:00.085192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.085063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27" event={"ID":"ca30e9ec-7602-4745-8fd9-1fba1e55129a","Type":"ContainerDied","Data":"46f2ad598be0b5b7bc4bf342baddbebe47b1b0b388a9dd7a0f1bd229fea5df73"}
Sep 06 01:09:00.085192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.085081    2575 scope.go:117] "RemoveContainer" containerID="fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d"
Sep 06 01:09:00.094646 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.094627    2575 scope.go:117] "RemoveContainer" containerID="fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d"
Sep 06 01:09:00.094941 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:00.094924    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d\": container with ID starting with fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d not found: ID does not exist" containerID="fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d"
Sep 06 01:09:00.094996 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.094949    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d"} err="failed to get container status \"fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d\": rpc error: code = NotFound desc = could not find container \"fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d\": container with ID starting with fa99aa317eafc76ade50ddeebca0efecf74a952a55144aeabdad39b286f6fc5d not found: ID does not exist"
Sep 06 01:09:00.106664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.106639    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"]
Sep 06 01:09:00.112094 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.112073    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-701b5b/test-model-701b5b-kserve-7858485947-8jh27"]
Sep 06 01:09:00.202866 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.202818    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca30e9ec-7602-4745-8fd9-1fba1e55129a" path="/var/lib/kubelet/pods/ca30e9ec-7602-4745-8fd9-1fba1e55129a/volumes"
Sep 06 01:09:00.278757 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.278712    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg" podUID="51c3d073-7877-4059-b1c8-e6f477af8436" containerName="main" containerID="cri-o://de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380" gracePeriod=45
Sep 06 01:09:00.544691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.544669    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:09:00.678802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.678709    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51c3d073-7877-4059-b1c8-e6f477af8436-tls-certs\") pod \"51c3d073-7877-4059-b1c8-e6f477af8436\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") "
Sep 06 01:09:00.678802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.678774    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84nss\" (UniqueName: \"kubernetes.io/projected/51c3d073-7877-4059-b1c8-e6f477af8436-kube-api-access-84nss\") pod \"51c3d073-7877-4059-b1c8-e6f477af8436\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") "
Sep 06 01:09:00.679045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.678819    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-home\") pod \"51c3d073-7877-4059-b1c8-e6f477af8436\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") "
Sep 06 01:09:00.679045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.678873    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-model-cache\") pod \"51c3d073-7877-4059-b1c8-e6f477af8436\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") "
Sep 06 01:09:00.679045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.678906    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-dshm\") pod \"51c3d073-7877-4059-b1c8-e6f477af8436\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") "
Sep 06 01:09:00.679045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.678950    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-tmp-dir\") pod \"51c3d073-7877-4059-b1c8-e6f477af8436\" (UID: \"51c3d073-7877-4059-b1c8-e6f477af8436\") "
Sep 06 01:09:00.679230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.679141    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-home" (OuterVolumeSpecName: "home") pod "51c3d073-7877-4059-b1c8-e6f477af8436" (UID: "51c3d073-7877-4059-b1c8-e6f477af8436"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:00.679293 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.679250    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-model-cache" (OuterVolumeSpecName: "model-cache") pod "51c3d073-7877-4059-b1c8-e6f477af8436" (UID: "51c3d073-7877-4059-b1c8-e6f477af8436"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:00.679347 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.679305    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.679347 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.679331    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "51c3d073-7877-4059-b1c8-e6f477af8436" (UID: "51c3d073-7877-4059-b1c8-e6f477af8436"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:00.681413 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.681377    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-dshm" (OuterVolumeSpecName: "dshm") pod "51c3d073-7877-4059-b1c8-e6f477af8436" (UID: "51c3d073-7877-4059-b1c8-e6f477af8436"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:00.681532 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.681443    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51c3d073-7877-4059-b1c8-e6f477af8436-kube-api-access-84nss" (OuterVolumeSpecName: "kube-api-access-84nss") pod "51c3d073-7877-4059-b1c8-e6f477af8436" (UID: "51c3d073-7877-4059-b1c8-e6f477af8436"). InnerVolumeSpecName "kube-api-access-84nss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:00.681532 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.681474    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51c3d073-7877-4059-b1c8-e6f477af8436-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "51c3d073-7877-4059-b1c8-e6f477af8436" (UID: "51c3d073-7877-4059-b1c8-e6f477af8436"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:09:00.742706 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.742665    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"]
Sep 06 01:09:00.777710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.777676    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:00.780559 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.780528    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.780630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.780566    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.780630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.780580    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51c3d073-7877-4059-b1c8-e6f477af8436-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.780630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.780592    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51c3d073-7877-4059-b1c8-e6f477af8436-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.780630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.780606    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84nss\" (UniqueName: \"kubernetes.io/projected/51c3d073-7877-4059-b1c8-e6f477af8436-kube-api-access-84nss\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:00.867879 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.866608    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"]
Sep 06 01:09:00.880881 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.872108    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr" podUID="75a579fc-9182-4549-80f6-e93d78106f13" containerName="istio-proxy" containerID="cri-o://fd0fa6d965fd5c2ba900c41811afe88571f2288753de6e1b8e7bdd7cc68bcf98" gracePeriod=30
Sep 06 01:09:00.884875 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:00.884774    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:01.091903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.091843    2575 generic.go:356] "Generic (PLEG): container finished" podID="51c3d073-7877-4059-b1c8-e6f477af8436" containerID="de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380" exitCode=0
Sep 06 01:09:01.091903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.091901    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"
Sep 06 01:09:01.092138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.091949    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg" event={"ID":"51c3d073-7877-4059-b1c8-e6f477af8436","Type":"ContainerDied","Data":"de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380"}
Sep 06 01:09:01.092138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.091993    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg" event={"ID":"51c3d073-7877-4059-b1c8-e6f477af8436","Type":"ContainerDied","Data":"803e1540cd11016d7d11324aeae1ee3965eaa82ac31537d0bcaf1688acfe3e5a"}
Sep 06 01:09:01.092138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.092018    2575 scope.go:117] "RemoveContainer" containerID="de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380"
Sep 06 01:09:01.101711 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.101692    2575 scope.go:117] "RemoveContainer" containerID="de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380"
Sep 06 01:09:01.102016 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:01.101991    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380\": container with ID starting with de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380 not found: ID does not exist" containerID="de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380"
Sep 06 01:09:01.102124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.102022    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380"} err="failed to get container status \"de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380\": rpc error: code = NotFound desc = could not find container \"de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380\": container with ID starting with de023323e158146d21115ca50084732ea6b807404a2187e0d9a77d002be88380 not found: ID does not exist"
Sep 06 01:09:01.114452 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.114421    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"]
Sep 06 01:09:01.119402 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.119379    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8d5457/test-model-8d5457-kserve-5dccf947d7-jmrtg"]
Sep 06 01:09:01.124996 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.124975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:01.239388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:01.239346    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:02.202568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:02.202529    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51c3d073-7877-4059-b1c8-e6f477af8436" path="/var/lib/kubelet/pods/51c3d073-7877-4059-b1c8-e6f477af8436/volumes"
Sep 06 01:09:02.557305 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:02.557195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:02.624398 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:02.624368    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:02.830151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:02.830074    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:03.166128 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166025    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"]
Sep 06 01:09:03.166629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166604    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca30e9ec-7602-4745-8fd9-1fba1e55129a" containerName="main"
Sep 06 01:09:03.166735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166631    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca30e9ec-7602-4745-8fd9-1fba1e55129a" containerName="main"
Sep 06 01:09:03.166735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166646    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51c3d073-7877-4059-b1c8-e6f477af8436" containerName="main"
Sep 06 01:09:03.166735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166655    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51c3d073-7877-4059-b1c8-e6f477af8436" containerName="main"
Sep 06 01:09:03.166939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166753    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="51c3d073-7877-4059-b1c8-e6f477af8436" containerName="main"
Sep 06 01:09:03.166939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.166768    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca30e9ec-7602-4745-8fd9-1fba1e55129a" containerName="main"
Sep 06 01:09:03.171417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.171380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:03.175529 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.175504    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:09:03.175952 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.175543    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:09:03.180370 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.180336    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"]
Sep 06 01:09:03.208608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.208576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef322b89-da9b-49f3-8650-d117511ed62d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz\" (UID: \"ef322b89-da9b-49f3-8650-d117511ed62d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:03.309756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.309717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef322b89-da9b-49f3-8650-d117511ed62d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz\" (UID: \"ef322b89-da9b-49f3-8650-d117511ed62d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:03.310408 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.310383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef322b89-da9b-49f3-8650-d117511ed62d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz\" (UID: \"ef322b89-da9b-49f3-8650-d117511ed62d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:03.348582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.348545    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:03.488652 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.488613    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:03.763403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:03.763293    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"]
Sep 06 01:09:03.778903 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:09:03.778823    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef322b89_da9b_49f3_8650_d117511ed62d.slice/crio-17e96c61363507e535cdb387a2ee6b9fa6744926c5dc5c6f75e09442e1030b8c WatchSource:0}: Error finding container 17e96c61363507e535cdb387a2ee6b9fa6744926c5dc5c6f75e09442e1030b8c: Status 404 returned error can't find the container with id 17e96c61363507e535cdb387a2ee6b9fa6744926c5dc5c6f75e09442e1030b8c
Sep 06 01:09:04.118946 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:04.118907    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz" event={"ID":"ef322b89-da9b-49f3-8650-d117511ed62d","Type":"ContainerStarted","Data":"17e96c61363507e535cdb387a2ee6b9fa6744926c5dc5c6f75e09442e1030b8c"}
Sep 06 01:09:05.049441 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:05.049366    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:09:05.127572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:05.127536    2575 generic.go:356] "Generic (PLEG): container finished" podID="ef322b89-da9b-49f3-8650-d117511ed62d" containerID="29221454570a502ec32ae20371407be539c14c4f27827baf8838d5cb37bd605d" exitCode=0
Sep 06 01:09:05.127703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:05.127616    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz" event={"ID":"ef322b89-da9b-49f3-8650-d117511ed62d","Type":"ContainerDied","Data":"29221454570a502ec32ae20371407be539c14c4f27827baf8838d5cb37bd605d"}
Sep 06 01:09:06.136467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.136403    2575 generic.go:356] "Generic (PLEG): container finished" podID="75a579fc-9182-4549-80f6-e93d78106f13" containerID="fd0fa6d965fd5c2ba900c41811afe88571f2288753de6e1b8e7bdd7cc68bcf98" exitCode=0
Sep 06 01:09:06.136782 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.136450    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr" event={"ID":"75a579fc-9182-4549-80f6-e93d78106f13","Type":"ContainerDied","Data":"fd0fa6d965fd5c2ba900c41811afe88571f2288753de6e1b8e7bdd7cc68bcf98"}
Sep 06 01:09:06.136782 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.136529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr" event={"ID":"75a579fc-9182-4549-80f6-e93d78106f13","Type":"ContainerDied","Data":"3a0339b727b1a74234bde16bb2c8d4ea59f50a3c5f40983f971dd8cab654ee15"}
Sep 06 01:09:06.136782 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.136556    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a0339b727b1a74234bde16bb2c8d4ea59f50a3c5f40983f971dd8cab654ee15"
Sep 06 01:09:06.151672 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.151647    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:09:06.235381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235352    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-socket\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.235549 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235394    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a579fc-9182-4549-80f6-e93d78106f13-istio-podinfo\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.235549 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235429    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-credential-socket\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.235712 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235681    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-certs\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.235712 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235675    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:06.235924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235716    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:06.235924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235728    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-istio-token\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.235924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235785    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts7l5\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-kube-api-access-ts7l5\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.235924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235834    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a579fc-9182-4549-80f6-e93d78106f13-istiod-ca-cert\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.236143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235940    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-data\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.236143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235949    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:06.236143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.235968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-envoy\") pod \"75a579fc-9182-4549-80f6-e93d78106f13\" (UID: \"75a579fc-9182-4549-80f6-e93d78106f13\") "
Sep 06 01:09:06.236325 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.236308    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.236378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.236333    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.236378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.236347    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.236378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.236345    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-data" (OuterVolumeSpecName: "istio-data") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:06.236378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.236349    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a579fc-9182-4549-80f6-e93d78106f13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:09:06.238632 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.238574    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-kube-api-access-ts7l5" (OuterVolumeSpecName: "kube-api-access-ts7l5") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "kube-api-access-ts7l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:06.238983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.238958    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:06.239321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.239298    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75a579fc-9182-4549-80f6-e93d78106f13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:09:06.239633 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.239609    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-istio-token" (OuterVolumeSpecName: "istio-token") pod "75a579fc-9182-4549-80f6-e93d78106f13" (UID: "75a579fc-9182-4549-80f6-e93d78106f13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:06.254003 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.253970    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:06.279765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.279739    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:06.337224 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337187    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef322b89-da9b-49f3-8650-d117511ed62d-maas-api-service-ca\") pod \"ef322b89-da9b-49f3-8650-d117511ed62d\" (UID: \"ef322b89-da9b-49f3-8650-d117511ed62d\") "
Sep 06 01:09:06.337527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337508    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.337568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337535    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a579fc-9182-4549-80f6-e93d78106f13-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.337568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337548    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a579fc-9182-4549-80f6-e93d78106f13-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.337568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337562    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.337670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337578    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts7l5\" (UniqueName: \"kubernetes.io/projected/75a579fc-9182-4549-80f6-e93d78106f13-kube-api-access-ts7l5\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.337670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337592    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a579fc-9182-4549-80f6-e93d78106f13-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:06.337670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.337602    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef322b89-da9b-49f3-8650-d117511ed62d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef322b89-da9b-49f3-8650-d117511ed62d" (UID: "ef322b89-da9b-49f3-8650-d117511ed62d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:09:06.438370 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:06.438285    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef322b89-da9b-49f3-8650-d117511ed62d-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:07.143432 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:07.143356    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"
Sep 06 01:09:07.143790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:07.143348    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz" event={"ID":"ef322b89-da9b-49f3-8650-d117511ed62d","Type":"ContainerDied","Data":"17e96c61363507e535cdb387a2ee6b9fa6744926c5dc5c6f75e09442e1030b8c"}
Sep 06 01:09:07.143790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:07.143484    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17e96c61363507e535cdb387a2ee6b9fa6744926c5dc5c6f75e09442e1030b8c"
Sep 06 01:09:07.143790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:07.143365    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"
Sep 06 01:09:07.171153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:07.171122    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"]
Sep 06 01:09:07.176385 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:07.176359    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-fb4286-openshift-default-856c456576-z5dqr"]
Sep 06 01:09:07.182512 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:07.182482    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef322b89_da9b_49f3_8650_d117511ed62d.slice\": RecentStats: unable to find data in memory cache]"
Sep 06 01:09:08.202658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:08.202622    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a579fc-9182-4549-80f6-e93d78106f13" path="/var/lib/kubelet/pods/75a579fc-9182-4549-80f6-e93d78106f13/volumes"
Sep 06 01:09:09.156519 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:09.156487    2575 generic.go:356] "Generic (PLEG): container finished" podID="aaad36fd-f812-462c-809f-95ca8423159b" containerID="3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d" exitCode=0
Sep 06 01:09:09.156718 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:09.156554    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" event={"ID":"aaad36fd-f812-462c-809f-95ca8423159b","Type":"ContainerDied","Data":"3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"}
Sep 06 01:09:09.157037 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:09.157020    2575 scope.go:117] "RemoveContainer" containerID="3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"
Sep 06 01:09:09.158081 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:09.158054    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:09.158146 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:09.158133    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:09.158183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:09.158169    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:10.172721 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:10.172682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" event={"ID":"aaad36fd-f812-462c-809f-95ca8423159b","Type":"ContainerStarted","Data":"e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862"}
Sep 06 01:09:10.542483 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:10.542448    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:09:10.542657 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:10.542622    2575 patch_prober.go:28] interesting pod/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 06 01:09:10.542701 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:10.542669    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 06 01:09:11.543243 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:11.543206    2575 patch_prober.go:28] interesting pod/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 06 01:09:11.543662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:11.543260    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 06 01:09:12.542604 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:12.542566    2575 patch_prober.go:28] interesting pod/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 06 01:09:12.542784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:12.542618    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 06 01:09:13.308241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:13.308209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:13.542947 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:13.542909    2575 patch_prober.go:28] interesting pod/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 06 01:09:13.543116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:13.542961    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 06 01:09:13.879935 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:13.879874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:13.976883 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:13.974584    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:14.068871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.068814    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:14.177727 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.177642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:14.229224 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.229191    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"]
Sep 06 01:09:14.235485 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.235459    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8d5457-thfgz"]
Sep 06 01:09:14.271279 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.271249    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:14.542459 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.542425    2575 patch_prober.go:28] interesting pod/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 06 01:09:14.542940 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.542483    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 06 01:09:14.677299 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:14.677268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:15.546821 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:15.546792    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:09:15.547272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:15.547253    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:09:15.548046 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:15.548026    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:09:15.785949 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:15.785900    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht" podUID="bc72d384-44d6-4ddf-bfdb-007f083d21eb" containerName="main" containerID="cri-o://17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f" gracePeriod=45
Sep 06 01:09:16.028449 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.028418    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:09:16.132057 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.131973    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-dshm\") pod \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") "
Sep 06 01:09:16.132057 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132034    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xbm5\" (UniqueName: \"kubernetes.io/projected/bc72d384-44d6-4ddf-bfdb-007f083d21eb-kube-api-access-8xbm5\") pod \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") "
Sep 06 01:09:16.132280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132070    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tmp-dir\") pod \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") "
Sep 06 01:09:16.132280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132119    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tls-certs\") pod \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") "
Sep 06 01:09:16.132280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-home\") pod \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") "
Sep 06 01:09:16.132280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132183    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-model-cache\") pod \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\" (UID: \"bc72d384-44d6-4ddf-bfdb-007f083d21eb\") "
Sep 06 01:09:16.132479 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132389    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc72d384-44d6-4ddf-bfdb-007f083d21eb" (UID: "bc72d384-44d6-4ddf-bfdb-007f083d21eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:16.132479 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132428    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-home" (OuterVolumeSpecName: "home") pod "bc72d384-44d6-4ddf-bfdb-007f083d21eb" (UID: "bc72d384-44d6-4ddf-bfdb-007f083d21eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:16.132627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132607    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:16.132679 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132634    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:16.132679 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.132615    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc72d384-44d6-4ddf-bfdb-007f083d21eb" (UID: "bc72d384-44d6-4ddf-bfdb-007f083d21eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:16.134520 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.134499    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc72d384-44d6-4ddf-bfdb-007f083d21eb" (UID: "bc72d384-44d6-4ddf-bfdb-007f083d21eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:09:16.134520 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.134513    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc72d384-44d6-4ddf-bfdb-007f083d21eb-kube-api-access-8xbm5" (OuterVolumeSpecName: "kube-api-access-8xbm5") pod "bc72d384-44d6-4ddf-bfdb-007f083d21eb" (UID: "bc72d384-44d6-4ddf-bfdb-007f083d21eb"). InnerVolumeSpecName "kube-api-access-8xbm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:16.134627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.134514    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-dshm" (OuterVolumeSpecName: "dshm") pod "bc72d384-44d6-4ddf-bfdb-007f083d21eb" (UID: "bc72d384-44d6-4ddf-bfdb-007f083d21eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:16.152402 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.152379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:16.202351 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.202320    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef322b89-da9b-49f3-8650-d117511ed62d" path="/var/lib/kubelet/pods/ef322b89-da9b-49f3-8650-d117511ed62d/volumes"
Sep 06 01:09:16.218955 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.218918    2575 generic.go:356] "Generic (PLEG): container finished" podID="bc72d384-44d6-4ddf-bfdb-007f083d21eb" containerID="17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f" exitCode=0
Sep 06 01:09:16.219891 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.219784    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht" event={"ID":"bc72d384-44d6-4ddf-bfdb-007f083d21eb","Type":"ContainerDied","Data":"17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f"}
Sep 06 01:09:16.219891 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.219868    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"
Sep 06 01:09:16.220079 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.219845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht" event={"ID":"bc72d384-44d6-4ddf-bfdb-007f083d21eb","Type":"ContainerDied","Data":"69bdb443eecd30fffb9c2cd2cb4bc902007e73029077f211d58adbd59f5374d4"}
Sep 06 01:09:16.220079 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.219877    2575 scope.go:117] "RemoveContainer" containerID="17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f"
Sep 06 01:09:16.229759 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.229742    2575 scope.go:117] "RemoveContainer" containerID="17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f"
Sep 06 01:09:16.230048 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:16.230025    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f\": container with ID starting with 17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f not found: ID does not exist" containerID="17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f"
Sep 06 01:09:16.230113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.230051    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f"} err="failed to get container status \"17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f\": rpc error: code = NotFound desc = could not find container \"17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f\": container with ID starting with 17473c5cb81c14a312d714f7bac18e6343e178b3dff0ae7e443939b1c2e9db1f not found: ID does not exist"
Sep 06 01:09:16.233484 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.233470    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc72d384-44d6-4ddf-bfdb-007f083d21eb-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:16.233528 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.233488    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:16.233528 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.233501    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc72d384-44d6-4ddf-bfdb-007f083d21eb-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:16.233528 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.233514    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xbm5\" (UniqueName: \"kubernetes.io/projected/bc72d384-44d6-4ddf-bfdb-007f083d21eb-kube-api-access-8xbm5\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:16.237611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.237592    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"]
Sep 06 01:09:16.241534 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.241511    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-fb4286/noten-model-fb4286-kserve-5fb5fcc955-g7sht"]
Sep 06 01:09:16.900905 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:16.900873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:17.014044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:17.013994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:17.321063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:17.321034    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:18.201445 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:18.201416    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc72d384-44d6-4ddf-bfdb-007f083d21eb" path="/var/lib/kubelet/pods/bc72d384-44d6-4ddf-bfdb-007f083d21eb/volumes"
Sep 06 01:09:20.432965 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:20.432927    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:20.530089 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:20.530056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:20.747681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:20.747643    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:21.312843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:21.312809    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:21.637133 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:21.637051    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:21.832499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:21.832457    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:23.557808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.557771    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:23.621415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.621385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:23.982424 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982379    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"]
Sep 06 01:09:23.982742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982729    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc72d384-44d6-4ddf-bfdb-007f083d21eb" containerName="main"
Sep 06 01:09:23.982783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982743    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc72d384-44d6-4ddf-bfdb-007f083d21eb" containerName="main"
Sep 06 01:09:23.982783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982756    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef322b89-da9b-49f3-8650-d117511ed62d" containerName="revoke-keys"
Sep 06 01:09:23.982783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982763    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef322b89-da9b-49f3-8650-d117511ed62d" containerName="revoke-keys"
Sep 06 01:09:23.982901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982784    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a579fc-9182-4549-80f6-e93d78106f13" containerName="istio-proxy"
Sep 06 01:09:23.982901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982789    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a579fc-9182-4549-80f6-e93d78106f13" containerName="istio-proxy"
Sep 06 01:09:23.982901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982848    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef322b89-da9b-49f3-8650-d117511ed62d" containerName="revoke-keys"
Sep 06 01:09:23.982901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982872    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc72d384-44d6-4ddf-bfdb-007f083d21eb" containerName="main"
Sep 06 01:09:23.982901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.982878    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a579fc-9182-4549-80f6-e93d78106f13" containerName="istio-proxy"
Sep 06 01:09:23.985953 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.985937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:23.988581 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.988560    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:09:23.988706 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.988560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:09:23.994623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:23.994595    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"]
Sep 06 01:09:24.113189 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:24.113157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc11ee9a-06e6-475f-baab-16e343a4af00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-40d55b26-wwg28\" (UID: \"dc11ee9a-06e6-475f-baab-16e343a4af00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:24.214416 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:24.214381    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc11ee9a-06e6-475f-baab-16e343a4af00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-40d55b26-wwg28\" (UID: \"dc11ee9a-06e6-475f-baab-16e343a4af00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:24.215039 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:24.215014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc11ee9a-06e6-475f-baab-16e343a4af00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-40d55b26-wwg28\" (UID: \"dc11ee9a-06e6-475f-baab-16e343a4af00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:24.297451 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:24.297356    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:24.578067 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:24.577990    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"]
Sep 06 01:09:24.589089 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:09:24.589064    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc11ee9a_06e6_475f_baab_16e343a4af00.slice/crio-cf78c01c85f1992a834635a13ce0f1985828acb622bfe06700afe8ce6875275c WatchSource:0}: Error finding container cf78c01c85f1992a834635a13ce0f1985828acb622bfe06700afe8ce6875275c: Status 404 returned error can't find the container with id cf78c01c85f1992a834635a13ce0f1985828acb622bfe06700afe8ce6875275c
Sep 06 01:09:25.122008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:25.121961    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:25.209529 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:25.209501    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:25.272498 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:25.272472    2575 generic.go:356] "Generic (PLEG): container finished" podID="dc11ee9a-06e6-475f-baab-16e343a4af00" containerID="c68349c6e4bc3baead8ed11330947cf9366334d86d213b1d4cf2da0b5c404053" exitCode=0
Sep 06 01:09:25.272598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:25.272552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28" event={"ID":"dc11ee9a-06e6-475f-baab-16e343a4af00","Type":"ContainerDied","Data":"c68349c6e4bc3baead8ed11330947cf9366334d86d213b1d4cf2da0b5c404053"}
Sep 06 01:09:25.272641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:25.272606    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28" event={"ID":"dc11ee9a-06e6-475f-baab-16e343a4af00","Type":"ContainerStarted","Data":"cf78c01c85f1992a834635a13ce0f1985828acb622bfe06700afe8ce6875275c"}
Sep 06 01:09:26.341671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.341641    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:26.410624 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.410526    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:26.424745 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.424720    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:26.535199 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.535117    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc11ee9a-06e6-475f-baab-16e343a4af00-maas-api-service-ca\") pod \"dc11ee9a-06e6-475f-baab-16e343a4af00\" (UID: \"dc11ee9a-06e6-475f-baab-16e343a4af00\") "
Sep 06 01:09:26.535499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.535478    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc11ee9a-06e6-475f-baab-16e343a4af00-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc11ee9a-06e6-475f-baab-16e343a4af00" (UID: "dc11ee9a-06e6-475f-baab-16e343a4af00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:09:26.636133 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.636106    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc11ee9a-06e6-475f-baab-16e343a4af00-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:26.693005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.692973    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:26.794271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.794170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:26.909916 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:26.909879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:27.285593 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:27.285566    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"
Sep 06 01:09:27.285762 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:27.285607    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28" event={"ID":"dc11ee9a-06e6-475f-baab-16e343a4af00","Type":"ContainerDied","Data":"cf78c01c85f1992a834635a13ce0f1985828acb622bfe06700afe8ce6875275c"}
Sep 06 01:09:27.285762 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:27.285652    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf78c01c85f1992a834635a13ce0f1985828acb622bfe06700afe8ce6875275c"
Sep 06 01:09:34.093521 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.093466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:34.704397 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.704368    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:34.732036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.731997    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"]
Sep 06 01:09:34.732265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.732242    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9" podUID="1b535f3b-443e-49e2-9e54-81828675a413" containerName="payload-processing" containerID="cri-o://deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2" gracePeriod=30
Sep 06 01:09:34.805698 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.805664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:34.889727 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.889695    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"]
Sep 06 01:09:34.891956 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.891927    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40d55b26-wwg28"]
Sep 06 01:09:34.892081 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.892024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:34.957144 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.957060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:34.984271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:34.984250    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:09:35.114597 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.114566    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b535f3b-443e-49e2-9e54-81828675a413-plugins-config-volume\") pod \"1b535f3b-443e-49e2-9e54-81828675a413\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") "
Sep 06 01:09:35.115084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.114684    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h56qh\" (UniqueName: \"kubernetes.io/projected/1b535f3b-443e-49e2-9e54-81828675a413-kube-api-access-h56qh\") pod \"1b535f3b-443e-49e2-9e54-81828675a413\" (UID: \"1b535f3b-443e-49e2-9e54-81828675a413\") "
Sep 06 01:09:35.115084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.115042    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b535f3b-443e-49e2-9e54-81828675a413-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b535f3b-443e-49e2-9e54-81828675a413" (UID: "1b535f3b-443e-49e2-9e54-81828675a413"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:09:35.116865 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.116820    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b535f3b-443e-49e2-9e54-81828675a413-kube-api-access-h56qh" (OuterVolumeSpecName: "kube-api-access-h56qh") pod "1b535f3b-443e-49e2-9e54-81828675a413" (UID: "1b535f3b-443e-49e2-9e54-81828675a413"). InnerVolumeSpecName "kube-api-access-h56qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:35.215825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.215751    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h56qh\" (UniqueName: \"kubernetes.io/projected/1b535f3b-443e-49e2-9e54-81828675a413-kube-api-access-h56qh\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:35.215825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.215778    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b535f3b-443e-49e2-9e54-81828675a413-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:35.254376 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.254347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:35.333606 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.333566    2575 generic.go:356] "Generic (PLEG): container finished" podID="1b535f3b-443e-49e2-9e54-81828675a413" containerID="deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2" exitCode=0
Sep 06 01:09:35.333606 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.333609    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"
Sep 06 01:09:35.333872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.333653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9" event={"ID":"1b535f3b-443e-49e2-9e54-81828675a413","Type":"ContainerDied","Data":"deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2"}
Sep 06 01:09:35.333872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.333710    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9" event={"ID":"1b535f3b-443e-49e2-9e54-81828675a413","Type":"ContainerDied","Data":"a2e5d2395e2814ea53d08184308218e6e4338dff2ef7b9f5ed24e3f8bca046ef"}
Sep 06 01:09:35.333872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.333741    2575 scope.go:117] "RemoveContainer" containerID="deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2"
Sep 06 01:09:35.343601 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.343586    2575 scope.go:117] "RemoveContainer" containerID="deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2"
Sep 06 01:09:35.343870 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:35.343829    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2\": container with ID starting with deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2 not found: ID does not exist" containerID="deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2"
Sep 06 01:09:35.343931 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.343876    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2"} err="failed to get container status \"deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2\": rpc error: code = NotFound desc = could not find container \"deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2\": container with ID starting with deefee035d1a6af12cb9b2a607ef8686e307abe8bf2282f6b4f3609f91cce2d2 not found: ID does not exist"
Sep 06 01:09:35.355450 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.355429    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"]
Sep 06 01:09:35.359101 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:35.359077    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-40d55b26-787b9545d9-bftk9"]
Sep 06 01:09:36.203181 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:36.203150    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b535f3b-443e-49e2-9e54-81828675a413" path="/var/lib/kubelet/pods/1b535f3b-443e-49e2-9e54-81828675a413/volumes"
Sep 06 01:09:36.203545 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:36.203475    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc11ee9a-06e6-475f-baab-16e343a4af00" path="/var/lib/kubelet/pods/dc11ee9a-06e6-475f-baab-16e343a4af00/volumes"
Sep 06 01:09:41.081409 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.081371    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:41.157311 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.157278    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:41.331976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.331903    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"]
Sep 06 01:09:41.332137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.332109    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" containerID="cri-o://e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862" gracePeriod=30
Sep 06 01:09:41.363644 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.363606    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:41.703768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.703733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:41.801622 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.801584    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:41.938653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:41.938621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:43.297979 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:43.297944    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:45.549351 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:45.549321    2575 patch_prober.go:28] interesting pod/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 01:09:45.549709 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:45.549384    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 01:09:46.586436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.586412    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:09:46.615812 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-data\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.615812 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615809    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-envoy\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-istio-token\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615879    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-credential-socket\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615917    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cpdr\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-kube-api-access-6cpdr\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615946    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad36fd-f812-462c-809f-95ca8423159b-istio-podinfo\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.615975    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-certs\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616011    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad36fd-f812-462c-809f-95ca8423159b-istiod-ca-cert\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616053    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-socket\") pod \"aaad36fd-f812-462c-809f-95ca8423159b\" (UID: \"aaad36fd-f812-462c-809f-95ca8423159b\") "
Sep 06 01:09:46.616381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616078    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-data" (OuterVolumeSpecName: "istio-data") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:46.616381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616292    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:46.616381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616306    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:46.616381 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616317    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.616640 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616495    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:46.616640 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.616543    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaad36fd-f812-462c-809f-95ca8423159b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:09:46.618292 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.618259    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-istio-token" (OuterVolumeSpecName: "istio-token") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:46.618746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.618722    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-kube-api-access-6cpdr" (OuterVolumeSpecName: "kube-api-access-6cpdr") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "kube-api-access-6cpdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:09:46.618906 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.618876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:09:46.619008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.618965    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aaad36fd-f812-462c-809f-95ca8423159b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aaad36fd-f812-462c-809f-95ca8423159b" (UID: "aaad36fd-f812-462c-809f-95ca8423159b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:09:46.717641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717615    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717637    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717646    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717656    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cpdr\" (UniqueName: \"kubernetes.io/projected/aaad36fd-f812-462c-809f-95ca8423159b-kube-api-access-6cpdr\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717666    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaad36fd-f812-462c-809f-95ca8423159b-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717675    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717683    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaad36fd-f812-462c-809f-95ca8423159b-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:46.717833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:46.717689    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaad36fd-f812-462c-809f-95ca8423159b-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:09:47.404596 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.404563    2575 generic.go:356] "Generic (PLEG): container finished" podID="aaad36fd-f812-462c-809f-95ca8423159b" containerID="e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862" exitCode=0
Sep 06 01:09:47.404879 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.404606    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"
Sep 06 01:09:47.404879 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.404650    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" event={"ID":"aaad36fd-f812-462c-809f-95ca8423159b","Type":"ContainerDied","Data":"e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862"}
Sep 06 01:09:47.404879 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.404700    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz" event={"ID":"aaad36fd-f812-462c-809f-95ca8423159b","Type":"ContainerDied","Data":"24838905d46e37af1e39152eef1168d99eb53b5f51c0f847f74130f733de7e73"}
Sep 06 01:09:47.404879 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.404721    2575 scope.go:117] "RemoveContainer" containerID="e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862"
Sep 06 01:09:47.414417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.414401    2575 scope.go:117] "RemoveContainer" containerID="3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"
Sep 06 01:09:47.422694 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.422679    2575 scope.go:117] "RemoveContainer" containerID="e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862"
Sep 06 01:09:47.423006 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:47.422980    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862\": container with ID starting with e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862 not found: ID does not exist" containerID="e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862"
Sep 06 01:09:47.423113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.423017    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862"} err="failed to get container status \"e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862\": rpc error: code = NotFound desc = could not find container \"e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862\": container with ID starting with e8a050ef98c9abae97b97d3beb5884b6f1a1da656c395a0b55c5bb94dcdde862 not found: ID does not exist"
Sep 06 01:09:47.423113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.423033    2575 scope.go:117] "RemoveContainer" containerID="3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"
Sep 06 01:09:47.423312 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:09:47.423282    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d\": container with ID starting with 3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d not found: ID does not exist" containerID="3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"
Sep 06 01:09:47.423404 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.423323    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d"} err="failed to get container status \"3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d\": rpc error: code = NotFound desc = could not find container \"3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d\": container with ID starting with 3143dc195e50df531975aa66483ea1bc531e6605dd7744d713dad344de162d8d not found: ID does not exist"
Sep 06 01:09:47.426115 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.426096    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"]
Sep 06 01:09:47.431963 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.431941    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-40d55b26-openshift-default-7bb59b86d8-rpwqz"]
Sep 06 01:09:47.632881 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.632827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:47.736174 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.736139    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:47.833252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:47.833220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:48.015359 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:48.015284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:48.207765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:48.207735    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaad36fd-f812-462c-809f-95ca8423159b" path="/var/lib/kubelet/pods/aaad36fd-f812-462c-809f-95ca8423159b/volumes"
Sep 06 01:09:48.261781 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:48.261751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:50.135236 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.135206    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:50.470981 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.470955    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:50.766932 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.766836    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"]
Sep 06 01:09:50.767292 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767274    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy"
Sep 06 01:09:50.767379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767296    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy"
Sep 06 01:09:50.767379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767314    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy"
Sep 06 01:09:50.767379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767325    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy"
Sep 06 01:09:50.767537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767356    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc11ee9a-06e6-475f-baab-16e343a4af00" containerName="revoke-keys"
Sep 06 01:09:50.767537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767392    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc11ee9a-06e6-475f-baab-16e343a4af00" containerName="revoke-keys"
Sep 06 01:09:50.767537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767405    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b535f3b-443e-49e2-9e54-81828675a413" containerName="payload-processing"
Sep 06 01:09:50.767537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767414    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b535f3b-443e-49e2-9e54-81828675a413" containerName="payload-processing"
Sep 06 01:09:50.767537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767529    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy"
Sep 06 01:09:50.767769 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767542    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b535f3b-443e-49e2-9e54-81828675a413" containerName="payload-processing"
Sep 06 01:09:50.767769 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767554    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc11ee9a-06e6-475f-baab-16e343a4af00" containerName="revoke-keys"
Sep 06 01:09:50.767769 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.767566    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaad36fd-f812-462c-809f-95ca8423159b" containerName="istio-proxy"
Sep 06 01:09:50.772007 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.771985    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.774451 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.774431    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-470346-openshift-default-dockercfg-8rtpf\""
Sep 06 01:09:50.782097 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.782072    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"]
Sep 06 01:09:50.786174 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.786126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:50.852531 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe251-fa84-41b4-8b9b-97dd865d366d-istiod-ca-cert\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-certs\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-token\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-data\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-podinfo\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-envoy\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852957 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcn58\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-kube-api-access-wcn58\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852957 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852750    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-credential-socket\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.852957 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.852800    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-socket\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.953976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.953918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-podinfo\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.953976 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.953982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-envoy\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcn58\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-kube-api-access-wcn58\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-credential-socket\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954353 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-socket\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954353 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954296    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe251-fa84-41b4-8b9b-97dd865d366d-istiod-ca-cert\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954353 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954335    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-certs\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954525 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954365    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-token\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954525 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954416    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-data\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954629    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-credential-socket\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.954713 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.954635    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-socket\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.955086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.955066    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-certs\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.955175 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.955134    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-data\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.955346 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.955326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe251-fa84-41b4-8b9b-97dd865d366d-istiod-ca-cert\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.956391 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.956366    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-envoy\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.956662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.956644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-podinfo\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.970387 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.970363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-token\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:50.972829 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:50.972811    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcn58\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-kube-api-access-wcn58\") pod \"e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") " pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:51.083369 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.083281    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:51.360230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.360205    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"]
Sep 06 01:09:51.361531 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:09:51.361502    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58fe251_fa84_41b4_8b9b_97dd865d366d.slice/crio-c03844c51aa7289a36b1582cd112ad2dd6567a65ec8195fe2c80e46b63d36643 WatchSource:0}: Error finding container c03844c51aa7289a36b1582cd112ad2dd6567a65ec8195fe2c80e46b63d36643: Status 404 returned error can't find the container with id c03844c51aa7289a36b1582cd112ad2dd6567a65ec8195fe2c80e46b63d36643
Sep 06 01:09:51.363556 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.363524    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:51.363659 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.363604    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:51.363659 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.363649    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:51.379094 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.379075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:51.432293 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:51.432265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" event={"ID":"e58fe251-fa84-41b4-8b9b-97dd865d366d","Type":"ContainerStarted","Data":"c03844c51aa7289a36b1582cd112ad2dd6567a65ec8195fe2c80e46b63d36643"}
Sep 06 01:09:52.440010 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:52.439972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" event={"ID":"e58fe251-fa84-41b4-8b9b-97dd865d366d","Type":"ContainerStarted","Data":"3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"}
Sep 06 01:09:52.460543 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:52.460491    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podStartSLOduration=2.460475625 podStartE2EDuration="2.460475625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:09:52.459137129 +0000 UTC m=+1154.788495965" watchObservedRunningTime="2026-09-06 01:09:52.460475625 +0000 UTC m=+1154.789834510"
Sep 06 01:09:53.083932 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:53.083898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:53.088250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:53.088226    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:53.446405 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:53.446325    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:53.447406 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:53.447387    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:09:53.837487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:53.837458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:53.940104 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:53.940070    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:54.391300 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:54.391268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:54.491451 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:54.491423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:54.811284 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:54.811253    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:55.112376 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.112302    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"]
Sep 06 01:09:55.115946 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.115926    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.118232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.118208    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-e6b4e2-openshift-default-dockercfg-phxtv\""
Sep 06 01:09:55.127132 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.127111    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"]
Sep 06 01:09:55.133313 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.133292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:55.194557 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194525    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-envoy\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.194557 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194555    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e50a7323-6cad-4ae0-887b-67c813061e9f-istiod-ca-cert\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.194792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-socket\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.194792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-token\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.194792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bddq7\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-kube-api-access-bddq7\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.194792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194663    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-podinfo\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.195033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194791    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-certs\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.195033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-credential-socket\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.195033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.194897    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-data\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296291    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-podinfo\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-certs\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-credential-socket\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296424    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-data\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-envoy\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e50a7323-6cad-4ae0-887b-67c813061e9f-istiod-ca-cert\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-socket\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-token\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.296790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bddq7\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-kube-api-access-bddq7\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.297016 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.296878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-credential-socket\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.297094 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.297068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-certs\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.297250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.297226    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-data\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.297371 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.297352    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-socket\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.297464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.297448    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e50a7323-6cad-4ae0-887b-67c813061e9f-istiod-ca-cert\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.299481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.299463    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-envoy\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.299672 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.299653    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-podinfo\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.305092 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.305070    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bddq7\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-kube-api-access-bddq7\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.305193 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.305076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-token\") pod \"e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") " pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.427013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.426929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:55.435831 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.435800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:55.542004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.541972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:55.610018 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.609984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:55.703352 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.703328    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"]
Sep 06 01:09:55.704402 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:09:55.704374    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50a7323_6cad_4ae0_887b_67c813061e9f.slice/crio-02b84a4033af7859dd560f44e5ed2b4ae77e425c5c6d1c0ef0e320592476dc46 WatchSource:0}: Error finding container 02b84a4033af7859dd560f44e5ed2b4ae77e425c5c6d1c0ef0e320592476dc46: Status 404 returned error can't find the container with id 02b84a4033af7859dd560f44e5ed2b4ae77e425c5c6d1c0ef0e320592476dc46
Sep 06 01:09:55.706663 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.706628    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:55.706732 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.706718    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:55.706769 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:55.706758    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:09:56.045370 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:56.045295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:56.145153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:56.145115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:56.465561 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:56.465527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" event={"ID":"e50a7323-6cad-4ae0-887b-67c813061e9f","Type":"ContainerStarted","Data":"597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"}
Sep 06 01:09:56.465741 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:56.465588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" event={"ID":"e50a7323-6cad-4ae0-887b-67c813061e9f","Type":"ContainerStarted","Data":"02b84a4033af7859dd560f44e5ed2b4ae77e425c5c6d1c0ef0e320592476dc46"}
Sep 06 01:09:56.485527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:56.485487    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podStartSLOduration=1.48547234 podStartE2EDuration="1.48547234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:09:56.483178142 +0000 UTC m=+1158.812536994" watchObservedRunningTime="2026-09-06 01:09:56.48547234 +0000 UTC m=+1158.814831176"
Sep 06 01:09:57.428009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:57.427973    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:57.433057 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:57.433031    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:57.470834 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:57.470807    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:57.471717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:57.471699    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:09:58.526612 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:58.526586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:58.620249 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:58.620223    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:59.855668 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:59.855532    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:09:59.921477 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:09:59.921443    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:00.173297 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.173226    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:00.319489 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.319450    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:00.411977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.411937    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:00.417175 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.417149    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"]
Sep 06 01:10:00.420608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.420592    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.422883 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.422842    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-e6b4e2\""
Sep 06 01:10:00.430712 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.430687    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"]
Sep 06 01:10:00.448159 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.448131    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"]
Sep 06 01:10:00.451509 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.451495    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.453676 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.453659    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-e6b4e2-dockercfg-jxpg4\""
Sep 06 01:10:00.461457 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.461434    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"]
Sep 06 01:10:00.549968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.549923    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgtn6\" (UniqueName: \"kubernetes.io/projected/26375bb7-5b82-4863-b63c-cddfd30e1a59-kube-api-access-bgtn6\") pod \"payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.550154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.550007    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c93615d-fca7-4a33-b242-e2bf0319a39b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.550154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.550078    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9mxt\" (UniqueName: \"kubernetes.io/projected/9c93615d-fca7-4a33-b242-e2bf0319a39b-kube-api-access-b9mxt\") pod \"payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.550330 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.550249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26375bb7-5b82-4863-b63c-cddfd30e1a59-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.650746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.650712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26375bb7-5b82-4863-b63c-cddfd30e1a59-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.650957 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.650758    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgtn6\" (UniqueName: \"kubernetes.io/projected/26375bb7-5b82-4863-b63c-cddfd30e1a59-kube-api-access-bgtn6\") pod \"payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.650957 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.650898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c93615d-fca7-4a33-b242-e2bf0319a39b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.650957 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.650947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9mxt\" (UniqueName: \"kubernetes.io/projected/9c93615d-fca7-4a33-b242-e2bf0319a39b-kube-api-access-b9mxt\") pod \"payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.651490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.651465    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26375bb7-5b82-4863-b63c-cddfd30e1a59-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.651657 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.651638    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c93615d-fca7-4a33-b242-e2bf0319a39b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.659026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.659002    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgtn6\" (UniqueName: \"kubernetes.io/projected/26375bb7-5b82-4863-b63c-cddfd30e1a59-kube-api-access-bgtn6\") pod \"payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:00.659128 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.659009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9mxt\" (UniqueName: \"kubernetes.io/projected/9c93615d-fca7-4a33-b242-e2bf0319a39b-kube-api-access-b9mxt\") pod \"payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.732628 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.732603    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:00.762254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:00.762227    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:01.007573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.007544    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"]
Sep 06 01:10:01.009054 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:01.009029    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c93615d_fca7_4a33_b242_e2bf0319a39b.slice/crio-4e2d1cf6729948a597b657b1f4aea6430fec39804dbf94cf1f99b8604818f61a WatchSource:0}: Error finding container 4e2d1cf6729948a597b657b1f4aea6430fec39804dbf94cf1f99b8604818f61a: Status 404 returned error can't find the container with id 4e2d1cf6729948a597b657b1f4aea6430fec39804dbf94cf1f99b8604818f61a
Sep 06 01:10:01.048687 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.048661    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"]
Sep 06 01:10:01.049237 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:01.049211    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26375bb7_5b82_4863_b63c_cddfd30e1a59.slice/crio-1bf3d02f0e29ba4d33f66507260e38d7e3095cf29759a2145dae047ca5f7ddb9 WatchSource:0}: Error finding container 1bf3d02f0e29ba4d33f66507260e38d7e3095cf29759a2145dae047ca5f7ddb9: Status 404 returned error can't find the container with id 1bf3d02f0e29ba4d33f66507260e38d7e3095cf29759a2145dae047ca5f7ddb9
Sep 06 01:10:01.499310 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.499277    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg" event={"ID":"9c93615d-fca7-4a33-b242-e2bf0319a39b","Type":"ContainerStarted","Data":"682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc"}
Sep 06 01:10:01.499524 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.499336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg" event={"ID":"9c93615d-fca7-4a33-b242-e2bf0319a39b","Type":"ContainerStarted","Data":"4e2d1cf6729948a597b657b1f4aea6430fec39804dbf94cf1f99b8604818f61a"}
Sep 06 01:10:01.499524 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.499378    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:01.501130 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.501095    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs" event={"ID":"26375bb7-5b82-4863-b63c-cddfd30e1a59","Type":"ContainerStarted","Data":"3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f"}
Sep 06 01:10:01.501231 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.501151    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:01.501231 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.501169    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs" event={"ID":"26375bb7-5b82-4863-b63c-cddfd30e1a59","Type":"ContainerStarted","Data":"1bf3d02f0e29ba4d33f66507260e38d7e3095cf29759a2145dae047ca5f7ddb9"}
Sep 06 01:10:01.517740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.517697    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg" podStartSLOduration=1.5176823000000002 podStartE2EDuration="1.5176823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:10:01.515106527 +0000 UTC m=+1163.844465389" watchObservedRunningTime="2026-09-06 01:10:01.5176823 +0000 UTC m=+1163.847041137"
Sep 06 01:10:01.533118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:01.533072    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs" podStartSLOduration=1.533058566 podStartE2EDuration="1.533058566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:10:01.530649089 +0000 UTC m=+1163.860007936" watchObservedRunningTime="2026-09-06 01:10:01.533058566 +0000 UTC m=+1163.862417401"
Sep 06 01:10:09.488044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:09.487998    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 01:10:09.488430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:09.488074    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 01:10:10.581430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:10.581393    2575 generic.go:356] "Generic (PLEG): container finished" podID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerID="3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161" exitCode=0
Sep 06 01:10:10.581843 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:10.581450    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" event={"ID":"e58fe251-fa84-41b4-8b9b-97dd865d366d","Type":"ContainerDied","Data":"3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"}
Sep 06 01:10:10.581955 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:10.581885    2575 scope.go:117] "RemoveContainer" containerID="3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"
Sep 06 01:10:10.582983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:10.582957    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:10:10.583056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:10.583024    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:10:10.583113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:10.583055    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:10:11.611660 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:11.611624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" event={"ID":"e58fe251-fa84-41b4-8b9b-97dd865d366d","Type":"ContainerStarted","Data":"37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03"}
Sep 06 01:10:12.084693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:12.084644    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:10:12.084991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:12.084787    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 01:10:12.084991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:12.084906    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 01:10:12.510009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:12.509979    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:10:12.510297 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:12.510286    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:10:13.084494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:13.084455    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 01:10:13.084867 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:13.084523    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 01:10:14.084582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:14.084544    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 01:10:14.085012 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:14.084612    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 01:10:15.084467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:15.084423    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 01:10:15.084635 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:15.084496    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 01:10:16.084123 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:16.084087    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Sep 06 01:10:16.084332 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:16.084140    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Sep 06 01:10:17.087060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.086966    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:10:17.087494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.087286    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:10:17.087840 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.087823    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:10:17.454911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.454878    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:17.657696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.657668    2575 generic.go:356] "Generic (PLEG): container finished" podID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerID="597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0" exitCode=0
Sep 06 01:10:17.657899 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.657745    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" event={"ID":"e50a7323-6cad-4ae0-887b-67c813061e9f","Type":"ContainerDied","Data":"597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"}
Sep 06 01:10:17.658232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.658218    2575 scope.go:117] "RemoveContainer" containerID="597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"
Sep 06 01:10:17.659276 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.659249    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:10:17.659375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.659351    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:10:17.659437 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:17.659401    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:10:18.670392 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:18.670346    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" event={"ID":"e50a7323-6cad-4ae0-887b-67c813061e9f","Type":"ContainerStarted","Data":"787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d"}
Sep 06 01:10:19.386170 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:19.386133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:19.428070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:19.428016    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:10:19.428382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:19.428359    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 06 01:10:19.428465 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:19.428415    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 06 01:10:20.427271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:20.427234    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 06 01:10:20.427718 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:20.427285    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 06 01:10:20.706169 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:20.706144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:21.428094 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:21.428057    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 06 01:10:21.428481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:21.428110    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 06 01:10:22.427734 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:22.427696    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 06 01:10:22.427945 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:22.427752    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 06 01:10:23.361228 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.361190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:23.428160 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.428126    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 06 01:10:23.428385 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.428187    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 06 01:10:23.564447 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.564402    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"]
Sep 06 01:10:23.566987 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.566966    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.570065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.570022    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"default-dockercfg-tnbc6\""
Sep 06 01:10:23.570065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.570042    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"kube-root-ca.crt\""
Sep 06 01:10:23.570272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.570069    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"auth-test-model-5844a8-kserve-self-signed-certs\""
Sep 06 01:10:23.570272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.570021    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"openshift-service-ca.crt\""
Sep 06 01:10:23.577518 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.577493    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"]
Sep 06 01:10:23.587813 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.587757    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:23.687872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.687755    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-tmp-dir\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.687872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.687813    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8459b165-c3e8-4b24-9267-cb4e61d10eca-tls-certs\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.687872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.687834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-model-cache\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.687872 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.687874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-home\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.688230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.687963    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-dshm\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.688230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.688006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdwcf\" (UniqueName: \"kubernetes.io/projected/8459b165-c3e8-4b24-9267-cb4e61d10eca-kube-api-access-rdwcf\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.720495 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.720470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:23.788675 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.788629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-dshm\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.788912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.788680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdwcf\" (UniqueName: \"kubernetes.io/projected/8459b165-c3e8-4b24-9267-cb4e61d10eca-kube-api-access-rdwcf\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.788912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.788725    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-tmp-dir\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.788912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.788778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8459b165-c3e8-4b24-9267-cb4e61d10eca-tls-certs\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.788912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.788807    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-model-cache\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.788912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.788834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-home\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.789265 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.789237    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-model-cache\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.789334 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.789247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-home\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.789452 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.789434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-tmp-dir\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.791222 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.791195    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-dshm\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.791660 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.791633    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8459b165-c3e8-4b24-9267-cb4e61d10eca-tls-certs\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.796954 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.796931    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdwcf\" (UniqueName: \"kubernetes.io/projected/8459b165-c3e8-4b24-9267-cb4e61d10eca-kube-api-access-rdwcf\") pod \"auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:23.877074 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:23.877032    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:24.036610 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.036573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:24.155399 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.155373    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"]
Sep 06 01:10:24.156982 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:24.156954    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8459b165_c3e8_4b24_9267_cb4e61d10eca.slice/crio-6cd3be534b59779cc0952844cd5b653ebf1ee9088246619ae5e144080056960c WatchSource:0}: Error finding container 6cd3be534b59779cc0952844cd5b653ebf1ee9088246619ae5e144080056960c: Status 404 returned error can't find the container with id 6cd3be534b59779cc0952844cd5b653ebf1ee9088246619ae5e144080056960c
Sep 06 01:10:24.221204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.221155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:24.431546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.431464    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:10:24.431983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.431916    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:10:24.432649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.432631    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:10:24.711998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.711961    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf" event={"ID":"8459b165-c3e8-4b24-9267-cb4e61d10eca","Type":"ContainerStarted","Data":"a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6"}
Sep 06 01:10:24.712191 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.712012    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf" event={"ID":"8459b165-c3e8-4b24-9267-cb4e61d10eca","Type":"ContainerStarted","Data":"6cd3be534b59779cc0952844cd5b653ebf1ee9088246619ae5e144080056960c"}
Sep 06 01:10:24.731192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:24.731145    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf" podStartSLOduration=1.731129917 podStartE2EDuration="1.731129917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:10:24.728249278 +0000 UTC m=+1187.057608133" watchObservedRunningTime="2026-09-06 01:10:24.731129917 +0000 UTC m=+1187.060488753"
Sep 06 01:10:27.468031 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:27.467982    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:27.559588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:27.559551    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:28.687170 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:28.687134    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:28.785093 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:28.785056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:28.934013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:28.933985    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:29.172157 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.172128    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:29.266898 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.266847    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:29.342374 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.342339    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"]
Sep 06 01:10:29.344842 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.344820    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.347280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.347258    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-812148-dockercfg-6fl7t\""
Sep 06 01:10:29.347440 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.347278    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-812148\""
Sep 06 01:10:29.358555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.358525    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"]
Sep 06 01:10:29.434054 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.433978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbwpm\" (UniqueName: \"kubernetes.io/projected/e35434d4-78d5-428b-8075-99299c8bfda7-kube-api-access-tbwpm\") pod \"payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.434220 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.434053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e35434d4-78d5-428b-8075-99299c8bfda7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.534599 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.534568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbwpm\" (UniqueName: \"kubernetes.io/projected/e35434d4-78d5-428b-8075-99299c8bfda7-kube-api-access-tbwpm\") pod \"payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.534790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.534678    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e35434d4-78d5-428b-8075-99299c8bfda7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.535429 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.535409    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e35434d4-78d5-428b-8075-99299c8bfda7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.544236 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.544206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbwpm\" (UniqueName: \"kubernetes.io/projected/e35434d4-78d5-428b-8075-99299c8bfda7-kube-api-access-tbwpm\") pod \"payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.657475 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.657435    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:29.937598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:29.937553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"]
Sep 06 01:10:29.939140 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:29.939117    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35434d4_78d5_428b_8075_99299c8bfda7.slice/crio-c275b8a2ee619c416cd6329be9c88670154a4b1d5f6669ccefb6028af003dac0 WatchSource:0}: Error finding container c275b8a2ee619c416cd6329be9c88670154a4b1d5f6669ccefb6028af003dac0: Status 404 returned error can't find the container with id c275b8a2ee619c416cd6329be9c88670154a4b1d5f6669ccefb6028af003dac0
Sep 06 01:10:30.760409 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:30.760367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv" event={"ID":"e35434d4-78d5-428b-8075-99299c8bfda7","Type":"ContainerStarted","Data":"3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f"}
Sep 06 01:10:30.760617 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:30.760466    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv" event={"ID":"e35434d4-78d5-428b-8075-99299c8bfda7","Type":"ContainerStarted","Data":"c275b8a2ee619c416cd6329be9c88670154a4b1d5f6669ccefb6028af003dac0"}
Sep 06 01:10:30.760617 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:30.760512    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:30.783298 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:30.783243    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv" podStartSLOduration=1.7832284710000001 podStartE2EDuration="1.783228471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:10:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:10:30.78142846 +0000 UTC m=+1193.110787317" watchObservedRunningTime="2026-09-06 01:10:30.783228471 +0000 UTC m=+1193.112587307"
Sep 06 01:10:33.877362 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:33.877323    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:33.877362 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:33.877371    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:33.889787 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:33.889754    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:34.797942 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:34.797907    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:10:38.243717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:38.243688    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:10:38.246623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:38.246597    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:10:38.350119 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:38.350089    2575 scope.go:117] "RemoveContainer" containerID="d7a653f80a88cb9de390fa6b3474af743f3e746a5236d1e246546200d11b7c3d"
Sep 06 01:10:41.768192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:41.768159    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:10:49.203457 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.203422    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:49.876125 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.876092    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"]
Sep 06 01:10:49.881645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.881623    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:49.884080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.884054    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"kube-root-ca.crt\""
Sep 06 01:10:49.884768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.884748    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"default-dockercfg-pvm25\""
Sep 06 01:10:49.884920 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.884779    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"openshift-service-ca.crt\""
Sep 06 01:10:49.884920 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.884753    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"auth-test-model-470346-kserve-self-signed-certs\""
Sep 06 01:10:49.889099 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.889077    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"]
Sep 06 01:10:49.905054 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:49.905023    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:50.017108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.017069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-model-cache\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.017331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.017125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc9f3434-3a40-40b0-b08f-780975591922-tls-certs\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.017331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.017194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-tmp-dir\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.017331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.017220    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-home\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.017331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.017242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x645f\" (UniqueName: \"kubernetes.io/projected/dc9f3434-3a40-40b0-b08f-780975591922-kube-api-access-x645f\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.017331 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.017312    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-dshm\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.118898 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.118836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-dshm\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.118924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-model-cache\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.118950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc9f3434-3a40-40b0-b08f-780975591922-tls-certs\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.118991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-tmp-dir\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.119005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-home\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.119027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x645f\" (UniqueName: \"kubernetes.io/projected/dc9f3434-3a40-40b0-b08f-780975591922-kube-api-access-x645f\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119398 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.119373    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-model-cache\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119458 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.119406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-home\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.119458 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.119440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-tmp-dir\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.121384 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.121364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-dshm\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.121498 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.121479    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc9f3434-3a40-40b0-b08f-780975591922-tls-certs\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.127128 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.127067    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x645f\" (UniqueName: \"kubernetes.io/projected/dc9f3434-3a40-40b0-b08f-780975591922-kube-api-access-x645f\") pod \"auth-test-model-470346-kserve-db44c4467-zm4k5\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") " pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.193028 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.193000    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:10:50.465889 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.465843    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"]
Sep 06 01:10:50.467164 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:50.467138    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc9f3434_3a40_40b0_b08f_780975591922.slice/crio-591b98f91c743cbaa3d51a8b8c7068dc98112f3c7fb423333e6839ed8009b794 WatchSource:0}: Error finding container 591b98f91c743cbaa3d51a8b8c7068dc98112f3c7fb423333e6839ed8009b794: Status 404 returned error can't find the container with id 591b98f91c743cbaa3d51a8b8c7068dc98112f3c7fb423333e6839ed8009b794
Sep 06 01:10:50.885026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.884938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5" event={"ID":"dc9f3434-3a40-40b0-b08f-780975591922","Type":"ContainerStarted","Data":"53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355"}
Sep 06 01:10:50.885026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.884975    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5" event={"ID":"dc9f3434-3a40-40b0-b08f-780975591922","Type":"ContainerStarted","Data":"591b98f91c743cbaa3d51a8b8c7068dc98112f3c7fb423333e6839ed8009b794"}
Sep 06 01:10:50.902614 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:50.902564    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5" podStartSLOduration=1.902548405 podStartE2EDuration="1.902548405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:10:50.900552182 +0000 UTC m=+1213.229911019" watchObservedRunningTime="2026-09-06 01:10:50.902548405 +0000 UTC m=+1213.231907242"
Sep 06 01:10:52.445792 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:52.445758    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:10:53.117626 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.117582    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:53.122059 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.122026    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:53.123629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.123603    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:53.124344 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.124323    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tmqsk\""
Sep 06 01:10:53.250067 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.250034    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr2qq\" (UniqueName: \"kubernetes.io/projected/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce-kube-api-access-nr2qq\") pod \"test-curl-7778-140167302399440\" (UID: \"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce\") " pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:53.350467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.350436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr2qq\" (UniqueName: \"kubernetes.io/projected/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce-kube-api-access-nr2qq\") pod \"test-curl-7778-140167302399440\" (UID: \"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce\") " pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:53.358670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.358642    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr2qq\" (UniqueName: \"kubernetes.io/projected/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce-kube-api-access-nr2qq\") pod \"test-curl-7778-140167302399440\" (UID: \"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce\") " pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:53.433274 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.433202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:53.703743 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.703720    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:53.705439 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:53.705412    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47e8d5c0_6d5c_47e8_806a_8f87de10a2ce.slice/crio-35905d1e686741a0be7f974d379c11e215f8cee6bb9f7ab556ceb7bd1ccc8a76 WatchSource:0}: Error finding container 35905d1e686741a0be7f974d379c11e215f8cee6bb9f7ab556ceb7bd1ccc8a76: Status 404 returned error can't find the container with id 35905d1e686741a0be7f974d379c11e215f8cee6bb9f7ab556ceb7bd1ccc8a76
Sep 06 01:10:53.706831 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.706812    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 01:10:53.903490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:53.903455    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302399440" event={"ID":"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce","Type":"ContainerStarted","Data":"35905d1e686741a0be7f974d379c11e215f8cee6bb9f7ab556ceb7bd1ccc8a76"}
Sep 06 01:10:54.911951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:54.911910    2575 generic.go:356] "Generic (PLEG): container finished" podID="47e8d5c0-6d5c-47e8-806a-8f87de10a2ce" containerID="38b0f4d80157b46d663fd4a085e97a090e1fe0349a7f2fa770987b19d2446363" exitCode=0
Sep 06 01:10:54.912371 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:54.911998    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302399440" event={"ID":"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce","Type":"ContainerDied","Data":"38b0f4d80157b46d663fd4a085e97a090e1fe0349a7f2fa770987b19d2446363"}
Sep 06 01:10:56.056608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.056586    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:56.075562 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.075539    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7778-140167302399440_47e8d5c0-6d5c-47e8-806a-8f87de10a2ce/test-curl-7778-140167302399440/0.log"
Sep 06 01:10:56.101409 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.101384    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:56.105161 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.105139    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:56.178671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.178605    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr2qq\" (UniqueName: \"kubernetes.io/projected/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce-kube-api-access-nr2qq\") pod \"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce\" (UID: \"47e8d5c0-6d5c-47e8-806a-8f87de10a2ce\") "
Sep 06 01:10:56.180748 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.180719    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce-kube-api-access-nr2qq" (OuterVolumeSpecName: "kube-api-access-nr2qq") pod "47e8d5c0-6d5c-47e8-806a-8f87de10a2ce" (UID: "47e8d5c0-6d5c-47e8-806a-8f87de10a2ce"). InnerVolumeSpecName "kube-api-access-nr2qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:10:56.202390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.202368    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47e8d5c0-6d5c-47e8-806a-8f87de10a2ce" path="/var/lib/kubelet/pods/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce/volumes"
Sep 06 01:10:56.234690 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.234665    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:56.235026 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.235013    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e8d5c0-6d5c-47e8-806a-8f87de10a2ce" containerName="test-curl-7778-140167302399440"
Sep 06 01:10:56.235076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.235027    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e8d5c0-6d5c-47e8-806a-8f87de10a2ce" containerName="test-curl-7778-140167302399440"
Sep 06 01:10:56.235118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.235100    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e8d5c0-6d5c-47e8-806a-8f87de10a2ce" containerName="test-curl-7778-140167302399440"
Sep 06 01:10:56.238061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.238047    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:56.244126 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.244106    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:56.279626 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.279603    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr2qq\" (UniqueName: \"kubernetes.io/projected/47e8d5c0-6d5c-47e8-806a-8f87de10a2ce-kube-api-access-nr2qq\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:10:56.380510 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.380481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f785p\" (UniqueName: \"kubernetes.io/projected/35445b08-4d75-4c57-a7bd-4dcc501012aa-kube-api-access-f785p\") pod \"test-curl-7778-140167302399440\" (UID: \"35445b08-4d75-4c57-a7bd-4dcc501012aa\") " pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:56.481552 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.481520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f785p\" (UniqueName: \"kubernetes.io/projected/35445b08-4d75-4c57-a7bd-4dcc501012aa-kube-api-access-f785p\") pod \"test-curl-7778-140167302399440\" (UID: \"35445b08-4d75-4c57-a7bd-4dcc501012aa\") " pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:56.489987 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.489959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f785p\" (UniqueName: \"kubernetes.io/projected/35445b08-4d75-4c57-a7bd-4dcc501012aa-kube-api-access-f785p\") pod \"test-curl-7778-140167302399440\" (UID: \"35445b08-4d75-4c57-a7bd-4dcc501012aa\") " pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:56.547755 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.547723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:56.823087 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.823063    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:56.824699 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:56.824674    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35445b08_4d75_4c57_a7bd_4dcc501012aa.slice/crio-82f9ed041a2484165556a49a9b5b2c29acdd2b891d6c0da8af4f1f73c5a9462a WatchSource:0}: Error finding container 82f9ed041a2484165556a49a9b5b2c29acdd2b891d6c0da8af4f1f73c5a9462a: Status 404 returned error can't find the container with id 82f9ed041a2484165556a49a9b5b2c29acdd2b891d6c0da8af4f1f73c5a9462a
Sep 06 01:10:56.925534 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.925500    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302399440" event={"ID":"35445b08-4d75-4c57-a7bd-4dcc501012aa","Type":"ContainerStarted","Data":"82f9ed041a2484165556a49a9b5b2c29acdd2b891d6c0da8af4f1f73c5a9462a"}
Sep 06 01:10:56.926804 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.926789    2575 scope.go:117] "RemoveContainer" containerID="38b0f4d80157b46d663fd4a085e97a090e1fe0349a7f2fa770987b19d2446363"
Sep 06 01:10:56.926915 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:56.926788    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:57.934403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:57.934371    2575 generic.go:356] "Generic (PLEG): container finished" podID="35445b08-4d75-4c57-a7bd-4dcc501012aa" containerID="61b4b443cd370a323f62e8dd473e0e26861190e60a080aa14402a3e5cc995bd2" exitCode=0
Sep 06 01:10:57.934838 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:57.934457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302399440" event={"ID":"35445b08-4d75-4c57-a7bd-4dcc501012aa","Type":"ContainerDied","Data":"61b4b443cd370a323f62e8dd473e0e26861190e60a080aa14402a3e5cc995bd2"}
Sep 06 01:10:59.069719 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.069694    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:59.090819 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.090791    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7778-140167302399440_35445b08-4d75-4c57-a7bd-4dcc501012aa/test-curl-7778-140167302399440/0.log"
Sep 06 01:10:59.118714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.118689    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:59.125180 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.125157    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7778-140167302399440"]
Sep 06 01:10:59.208491 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.208468    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f785p\" (UniqueName: \"kubernetes.io/projected/35445b08-4d75-4c57-a7bd-4dcc501012aa-kube-api-access-f785p\") pod \"35445b08-4d75-4c57-a7bd-4dcc501012aa\" (UID: \"35445b08-4d75-4c57-a7bd-4dcc501012aa\") "
Sep 06 01:10:59.210593 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.210560    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35445b08-4d75-4c57-a7bd-4dcc501012aa-kube-api-access-f785p" (OuterVolumeSpecName: "kube-api-access-f785p") pod "35445b08-4d75-4c57-a7bd-4dcc501012aa" (UID: "35445b08-4d75-4c57-a7bd-4dcc501012aa"). InnerVolumeSpecName "kube-api-access-f785p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:10:59.260984 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.260950    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:10:59.261348 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.261336    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35445b08-4d75-4c57-a7bd-4dcc501012aa" containerName="test-curl-7778-140167302399440"
Sep 06 01:10:59.261390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.261351    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="35445b08-4d75-4c57-a7bd-4dcc501012aa" containerName="test-curl-7778-140167302399440"
Sep 06 01:10:59.261430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.261416    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="35445b08-4d75-4c57-a7bd-4dcc501012aa" containerName="test-curl-7778-140167302399440"
Sep 06 01:10:59.264395 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.264380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:10:59.270723 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.270699    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:10:59.309977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.309921    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f785p\" (UniqueName: \"kubernetes.io/projected/35445b08-4d75-4c57-a7bd-4dcc501012aa-kube-api-access-f785p\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:10:59.411445 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.411399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kchkz\" (UniqueName: \"kubernetes.io/projected/24c50400-309b-43e8-b6d0-d64043b751f0-kube-api-access-kchkz\") pod \"test-curl-7778-140167302400016\" (UID: \"24c50400-309b-43e8-b6d0-d64043b751f0\") " pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:10:59.512106 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.512027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchkz\" (UniqueName: \"kubernetes.io/projected/24c50400-309b-43e8-b6d0-d64043b751f0-kube-api-access-kchkz\") pod \"test-curl-7778-140167302400016\" (UID: \"24c50400-309b-43e8-b6d0-d64043b751f0\") " pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:10:59.520363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.520338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchkz\" (UniqueName: \"kubernetes.io/projected/24c50400-309b-43e8-b6d0-d64043b751f0-kube-api-access-kchkz\") pod \"test-curl-7778-140167302400016\" (UID: \"24c50400-309b-43e8-b6d0-d64043b751f0\") " pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:10:59.574631 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.574605    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:10:59.846920 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.846896    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:10:59.848173 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:10:59.848145    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24c50400_309b_43e8_b6d0_d64043b751f0.slice/crio-3954e1a8acd69dd18f9d65d5ef759695cf76b5b6a874c51a9c6b39d7f1f39e29 WatchSource:0}: Error finding container 3954e1a8acd69dd18f9d65d5ef759695cf76b5b6a874c51a9c6b39d7f1f39e29: Status 404 returned error can't find the container with id 3954e1a8acd69dd18f9d65d5ef759695cf76b5b6a874c51a9c6b39d7f1f39e29
Sep 06 01:10:59.948730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.948705    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:10:59.948847 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.948736    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82f9ed041a2484165556a49a9b5b2c29acdd2b891d6c0da8af4f1f73c5a9462a"
Sep 06 01:10:59.950112 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:10:59.950089    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302400016" event={"ID":"24c50400-309b-43e8-b6d0-d64043b751f0","Type":"ContainerStarted","Data":"3954e1a8acd69dd18f9d65d5ef759695cf76b5b6a874c51a9c6b39d7f1f39e29"}
Sep 06 01:10:59.959167 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:10:59.959142    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7778-140167302399440\" is forbidden: User \"system:node:ip-10-0-129-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-5.ec2.internal' and this object" podUID="35445b08-4d75-4c57-a7bd-4dcc501012aa" pod="opendatahub/test-curl-7778-140167302399440"
Sep 06 01:11:00.194010 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.193908    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:11:00.194010 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.193970    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:11:00.208711 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.208675    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35445b08-4d75-4c57-a7bd-4dcc501012aa" path="/var/lib/kubelet/pods/35445b08-4d75-4c57-a7bd-4dcc501012aa/volumes"
Sep 06 01:11:00.210369 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.210341    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:11:00.957752 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.957723    2575 generic.go:356] "Generic (PLEG): container finished" podID="24c50400-309b-43e8-b6d0-d64043b751f0" containerID="8813ef01dbdda37699f19a48d2545b737e60e80431adb3f1818c8e6c7c89d4d6" exitCode=0
Sep 06 01:11:00.957922 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.957808    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302400016" event={"ID":"24c50400-309b-43e8-b6d0-d64043b751f0","Type":"ContainerDied","Data":"8813ef01dbdda37699f19a48d2545b737e60e80431adb3f1818c8e6c7c89d4d6"}
Sep 06 01:11:00.969467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:00.969446    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:11:02.097590 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.097564    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.118506 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.118475    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7778-140167302400016_24c50400-309b-43e8-b6d0-d64043b751f0/test-curl-7778-140167302400016/0.log"
Sep 06 01:11:02.144362 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.144318    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:02.148041 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.148020    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:02.237945 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.237840    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kchkz\" (UniqueName: \"kubernetes.io/projected/24c50400-309b-43e8-b6d0-d64043b751f0-kube-api-access-kchkz\") pod \"24c50400-309b-43e8-b6d0-d64043b751f0\" (UID: \"24c50400-309b-43e8-b6d0-d64043b751f0\") "
Sep 06 01:11:02.239998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.239973    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c50400-309b-43e8-b6d0-d64043b751f0-kube-api-access-kchkz" (OuterVolumeSpecName: "kube-api-access-kchkz") pod "24c50400-309b-43e8-b6d0-d64043b751f0" (UID: "24c50400-309b-43e8-b6d0-d64043b751f0"). InnerVolumeSpecName "kube-api-access-kchkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:11:02.275349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.275316    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:02.275693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.275680    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c50400-309b-43e8-b6d0-d64043b751f0" containerName="test-curl-7778-140167302400016"
Sep 06 01:11:02.275735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.275694    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c50400-309b-43e8-b6d0-d64043b751f0" containerName="test-curl-7778-140167302400016"
Sep 06 01:11:02.275778 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.275769    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c50400-309b-43e8-b6d0-d64043b751f0" containerName="test-curl-7778-140167302400016"
Sep 06 01:11:02.278768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.278752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.285137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.285110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:02.339308 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.339278    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kchkz\" (UniqueName: \"kubernetes.io/projected/24c50400-309b-43e8-b6d0-d64043b751f0-kube-api-access-kchkz\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:11:02.440275 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.440238    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghspl\" (UniqueName: \"kubernetes.io/projected/3ea711aa-1c6a-4f31-b523-307fe91589ac-kube-api-access-ghspl\") pod \"test-curl-7778-140167302400016\" (UID: \"3ea711aa-1c6a-4f31-b523-307fe91589ac\") " pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.541174 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.541090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghspl\" (UniqueName: \"kubernetes.io/projected/3ea711aa-1c6a-4f31-b523-307fe91589ac-kube-api-access-ghspl\") pod \"test-curl-7778-140167302400016\" (UID: \"3ea711aa-1c6a-4f31-b523-307fe91589ac\") " pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.548984 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.548955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghspl\" (UniqueName: \"kubernetes.io/projected/3ea711aa-1c6a-4f31-b523-307fe91589ac-kube-api-access-ghspl\") pod \"test-curl-7778-140167302400016\" (UID: \"3ea711aa-1c6a-4f31-b523-307fe91589ac\") " pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.589966 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.589917    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.860929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.860905    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:02.862508 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:11:02.862479    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ea711aa_1c6a_4f31_b523_307fe91589ac.slice/crio-c64878ca3c26c26aae3cab5547e3b81725dd370d68e6ea8f82450c516023ddf4 WatchSource:0}: Error finding container c64878ca3c26c26aae3cab5547e3b81725dd370d68e6ea8f82450c516023ddf4: Status 404 returned error can't find the container with id c64878ca3c26c26aae3cab5547e3b81725dd370d68e6ea8f82450c516023ddf4
Sep 06 01:11:02.970572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.970545    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:02.970572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.970567    2575 scope.go:117] "RemoveContainer" containerID="8813ef01dbdda37699f19a48d2545b737e60e80431adb3f1818c8e6c7c89d4d6"
Sep 06 01:11:02.972054 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:02.972028    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302400016" event={"ID":"3ea711aa-1c6a-4f31-b523-307fe91589ac","Type":"ContainerStarted","Data":"c64878ca3c26c26aae3cab5547e3b81725dd370d68e6ea8f82450c516023ddf4"}
Sep 06 01:11:03.979281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:03.979247    2575 generic.go:356] "Generic (PLEG): container finished" podID="3ea711aa-1c6a-4f31-b523-307fe91589ac" containerID="1b442f08fa39dc6cf20e16184196f04fe28280a28993b4726d91cfd7580c2b50" exitCode=0
Sep 06 01:11:03.979694 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:03.979339    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7778-140167302400016" event={"ID":"3ea711aa-1c6a-4f31-b523-307fe91589ac","Type":"ContainerDied","Data":"1b442f08fa39dc6cf20e16184196f04fe28280a28993b4726d91cfd7580c2b50"}
Sep 06 01:11:04.208159 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:04.208114    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c50400-309b-43e8-b6d0-d64043b751f0" path="/var/lib/kubelet/pods/24c50400-309b-43e8-b6d0-d64043b751f0/volumes"
Sep 06 01:11:05.115573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.115546    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:05.135531 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.135502    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7778-140167302400016_3ea711aa-1c6a-4f31-b523-307fe91589ac/test-curl-7778-140167302400016/0.log"
Sep 06 01:11:05.161487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.161460    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:05.167989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.167968    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7778-140167302400016"]
Sep 06 01:11:05.269151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.269069    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghspl\" (UniqueName: \"kubernetes.io/projected/3ea711aa-1c6a-4f31-b523-307fe91589ac-kube-api-access-ghspl\") pod \"3ea711aa-1c6a-4f31-b523-307fe91589ac\" (UID: \"3ea711aa-1c6a-4f31-b523-307fe91589ac\") "
Sep 06 01:11:05.271334 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.271307    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ea711aa-1c6a-4f31-b523-307fe91589ac-kube-api-access-ghspl" (OuterVolumeSpecName: "kube-api-access-ghspl") pod "3ea711aa-1c6a-4f31-b523-307fe91589ac" (UID: "3ea711aa-1c6a-4f31-b523-307fe91589ac"). InnerVolumeSpecName "kube-api-access-ghspl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:11:05.370483 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.370435    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghspl\" (UniqueName: \"kubernetes.io/projected/3ea711aa-1c6a-4f31-b523-307fe91589ac-kube-api-access-ghspl\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:11:05.800008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.799975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:05.993845 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.993817    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:05.994043 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:05.993896    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c64878ca3c26c26aae3cab5547e3b81725dd370d68e6ea8f82450c516023ddf4"
Sep 06 01:11:06.003735 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:11:06.003706    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7778-140167302400016\" is forbidden: User \"system:node:ip-10-0-129-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-5.ec2.internal' and this object" podUID="3ea711aa-1c6a-4f31-b523-307fe91589ac" pod="opendatahub/test-curl-7778-140167302400016"
Sep 06 01:11:06.145072 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:06.144972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:06.203050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:06.202999    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ea711aa-1c6a-4f31-b523-307fe91589ac" path="/var/lib/kubelet/pods/3ea711aa-1c6a-4f31-b523-307fe91589ac/volumes"
Sep 06 01:11:06.302994 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:06.302965    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:09.801486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:09.801454    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:09.900125 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:09.900083    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:11.161095 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.161050    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:11.262120 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.262072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:11.337438 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.337406    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:11.713172 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.713143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:11.803736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.803703    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:11.844058 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.844029    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"]
Sep 06 01:11:11.844428 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.844413    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ea711aa-1c6a-4f31-b523-307fe91589ac" containerName="test-curl-7778-140167302400016"
Sep 06 01:11:11.844428 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.844429    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ea711aa-1c6a-4f31-b523-307fe91589ac" containerName="test-curl-7778-140167302400016"
Sep 06 01:11:11.844560 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.844512    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ea711aa-1c6a-4f31-b523-307fe91589ac" containerName="test-curl-7778-140167302400016"
Sep 06 01:11:11.847769 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.847752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:11.850153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.850134    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7534e03d\""
Sep 06 01:11:11.850231 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.850134    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7534e03d-dockercfg-c8qxm\""
Sep 06 01:11:11.857884 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:11.857843    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"]
Sep 06 01:11:12.032535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.032449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3932d42d-3541-41e3-8424-77c154d7382e-plugins-config-volume\") pod \"payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") " pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.032535 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.032502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq2tz\" (UniqueName: \"kubernetes.io/projected/3932d42d-3541-41e3-8424-77c154d7382e-kube-api-access-hq2tz\") pod \"payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") " pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.133228 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.133194    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3932d42d-3541-41e3-8424-77c154d7382e-plugins-config-volume\") pod \"payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") " pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.133379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.133253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq2tz\" (UniqueName: \"kubernetes.io/projected/3932d42d-3541-41e3-8424-77c154d7382e-kube-api-access-hq2tz\") pod \"payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") " pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.133918 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.133900    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3932d42d-3541-41e3-8424-77c154d7382e-plugins-config-volume\") pod \"payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") " pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.141283 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.141257    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq2tz\" (UniqueName: \"kubernetes.io/projected/3932d42d-3541-41e3-8424-77c154d7382e-kube-api-access-hq2tz\") pod \"payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") " pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.158693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.158659    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:12.433250 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:12.433224    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"]
Sep 06 01:11:12.434793 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:11:12.434761    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3932d42d_3541_41e3_8424_77c154d7382e.slice/crio-02500c7e8810389ac130705186ffde3b7b939c45224bed07a265fd536ac023cf WatchSource:0}: Error finding container 02500c7e8810389ac130705186ffde3b7b939c45224bed07a265fd536ac023cf: Status 404 returned error can't find the container with id 02500c7e8810389ac130705186ffde3b7b939c45224bed07a265fd536ac023cf
Sep 06 01:11:13.038166 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:13.038132    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t" event={"ID":"3932d42d-3541-41e3-8424-77c154d7382e","Type":"ContainerStarted","Data":"0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d"}
Sep 06 01:11:13.038416 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:13.038194    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:13.038416 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:13.038211    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t" event={"ID":"3932d42d-3541-41e3-8424-77c154d7382e","Type":"ContainerStarted","Data":"02500c7e8810389ac130705186ffde3b7b939c45224bed07a265fd536ac023cf"}
Sep 06 01:11:13.058838 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:13.058775    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t" podStartSLOduration=2.058753149 podStartE2EDuration="2.058753149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:11:13.056737514 +0000 UTC m=+1235.386096373" watchObservedRunningTime="2026-09-06 01:11:13.058753149 +0000 UTC m=+1235.388111986"
Sep 06 01:11:13.175838 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:13.175804    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:24.047503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:24.047473    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:11:32.086962 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:32.086873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:34.711392 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:34.711360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:38.375984 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:38.375952    2575 scope.go:117] "RemoveContainer" containerID="3276d286d4d22d4820a0442a506629e7d13e89a137f35032235bd3333b710009"
Sep 06 01:11:41.028571 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:41.028503    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:41.857909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:41.857876    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:41.957799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:41.957772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:47.924143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.924107    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"]
Sep 06 01:11:47.929241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.929221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:47.932193 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.932172    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7534e03d\"/\"kube-root-ca.crt\""
Sep 06 01:11:47.932793 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.932774    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7534e03d\"/\"openshift-service-ca.crt\""
Sep 06 01:11:47.932998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.932978    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7534e03d\"/\"e2e-lifecycle-model-7534e03d-kserve-self-signed-certs\""
Sep 06 01:11:47.933203 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.933185    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7534e03d\"/\"default-dockercfg-pflxv\""
Sep 06 01:11:47.949314 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:47.949288    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"]
Sep 06 01:11:48.035318 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.035287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:11:48.071528 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.071502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-tmp-dir\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.071710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.071575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-model-cache\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.071710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.071605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-dshm\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.071710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.071639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj9hj\" (UniqueName: \"kubernetes.io/projected/4b61fdb4-f612-4f19-a620-30c16a27d2db-kube-api-access-vj9hj\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.071710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.071701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-home\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.071956 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.071778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b61fdb4-f612-4f19-a620-30c16a27d2db-tls-certs\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.172732 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.172703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-tmp-dir\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.172895 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.172790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-model-cache\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.172895 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.172834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-dshm\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.172977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.172911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj9hj\" (UniqueName: \"kubernetes.io/projected/4b61fdb4-f612-4f19-a620-30c16a27d2db-kube-api-access-vj9hj\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.172977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.172952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-home\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.173072 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.172994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b61fdb4-f612-4f19-a620-30c16a27d2db-tls-certs\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.173268 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.173244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-tmp-dir\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.173356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.173277    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-model-cache\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.173356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.173338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-home\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.175352 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.175300    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-dshm\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.175473 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.175456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b61fdb4-f612-4f19-a620-30c16a27d2db-tls-certs\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.181942 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.181909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj9hj\" (UniqueName: \"kubernetes.io/projected/4b61fdb4-f612-4f19-a620-30c16a27d2db-kube-api-access-vj9hj\") pod \"e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") " pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.238366 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.238347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:48.513368 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:48.513340    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"]
Sep 06 01:11:48.515359 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:11:48.515333    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b61fdb4_f612_4f19_a620_30c16a27d2db.slice/crio-29c5a78f620f12d1a36500a709e0f1dc2b0af985d12824e6cd2b7f91896f7e46 WatchSource:0}: Error finding container 29c5a78f620f12d1a36500a709e0f1dc2b0af985d12824e6cd2b7f91896f7e46: Status 404 returned error can't find the container with id 29c5a78f620f12d1a36500a709e0f1dc2b0af985d12824e6cd2b7f91896f7e46
Sep 06 01:11:49.261079 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:49.261042    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55" event={"ID":"4b61fdb4-f612-4f19-a620-30c16a27d2db","Type":"ContainerStarted","Data":"554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f"}
Sep 06 01:11:49.261079 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:49.261082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55" event={"ID":"4b61fdb4-f612-4f19-a620-30c16a27d2db","Type":"ContainerStarted","Data":"29c5a78f620f12d1a36500a709e0f1dc2b0af985d12824e6cd2b7f91896f7e46"}
Sep 06 01:11:49.279010 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:49.278952    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55" podStartSLOduration=2.278934033 podStartE2EDuration="2.278934033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:11:49.277751654 +0000 UTC m=+1271.607110507" watchObservedRunningTime="2026-09-06 01:11:49.278934033 +0000 UTC m=+1271.608292868"
Sep 06 01:11:58.239168 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:58.239130    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:58.239688 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:58.239180    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:58.251782 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:58.251756    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:11:58.331098 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:11:58.331067    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:12:17.219999 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:17.219966    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:17.281452 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:17.281415    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:29.809483 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:29.809447    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:29.873125 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:29.873095    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"]
Sep 06 01:12:29.873356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:29.873331    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t" podUID="3932d42d-3541-41e3-8424-77c154d7382e" containerName="payload-processing" containerID="cri-o://0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d" gracePeriod=30
Sep 06 01:12:29.903086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:29.903055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:29.967471 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:29.967430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:30.045315 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.045280    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:30.114323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.114293    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:30.125097 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.125081    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:12:30.274938 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.274906    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3932d42d-3541-41e3-8424-77c154d7382e-plugins-config-volume\") pod \"3932d42d-3541-41e3-8424-77c154d7382e\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") "
Sep 06 01:12:30.275097 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.274982    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq2tz\" (UniqueName: \"kubernetes.io/projected/3932d42d-3541-41e3-8424-77c154d7382e-kube-api-access-hq2tz\") pod \"3932d42d-3541-41e3-8424-77c154d7382e\" (UID: \"3932d42d-3541-41e3-8424-77c154d7382e\") "
Sep 06 01:12:30.275349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.275323    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3932d42d-3541-41e3-8424-77c154d7382e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3932d42d-3541-41e3-8424-77c154d7382e" (UID: "3932d42d-3541-41e3-8424-77c154d7382e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:12:30.277211 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.277187    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3932d42d-3541-41e3-8424-77c154d7382e-kube-api-access-hq2tz" (OuterVolumeSpecName: "kube-api-access-hq2tz") pod "3932d42d-3541-41e3-8424-77c154d7382e" (UID: "3932d42d-3541-41e3-8424-77c154d7382e"). InnerVolumeSpecName "kube-api-access-hq2tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:12:30.376076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.375990    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq2tz\" (UniqueName: \"kubernetes.io/projected/3932d42d-3541-41e3-8424-77c154d7382e-kube-api-access-hq2tz\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:30.376076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.376020    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3932d42d-3541-41e3-8424-77c154d7382e-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:30.516001 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.515963    2575 generic.go:356] "Generic (PLEG): container finished" podID="3932d42d-3541-41e3-8424-77c154d7382e" containerID="0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d" exitCode=0
Sep 06 01:12:30.516196 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.516022    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"
Sep 06 01:12:30.516196 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.516045    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t" event={"ID":"3932d42d-3541-41e3-8424-77c154d7382e","Type":"ContainerDied","Data":"0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d"}
Sep 06 01:12:30.516196 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.516117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t" event={"ID":"3932d42d-3541-41e3-8424-77c154d7382e","Type":"ContainerDied","Data":"02500c7e8810389ac130705186ffde3b7b939c45224bed07a265fd536ac023cf"}
Sep 06 01:12:30.516196 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.516146    2575 scope.go:117] "RemoveContainer" containerID="0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d"
Sep 06 01:12:30.525956 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.525936    2575 scope.go:117] "RemoveContainer" containerID="0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d"
Sep 06 01:12:30.526229 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:12:30.526207    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d\": container with ID starting with 0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d not found: ID does not exist" containerID="0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d"
Sep 06 01:12:30.526304 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.526239    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d"} err="failed to get container status \"0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d\": rpc error: code = NotFound desc = could not find container \"0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d\": container with ID starting with 0f1b0a2cf52e5d68b73352f06b760db3941be837ad70dca3b872a961568c8a7d not found: ID does not exist"
Sep 06 01:12:30.539048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.539026    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"]
Sep 06 01:12:30.543080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:30.543059    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7534e03d-75f5c9bfc-f7r8t"]
Sep 06 01:12:32.203005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:32.202971    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3932d42d-3541-41e3-8424-77c154d7382e" path="/var/lib/kubelet/pods/3932d42d-3541-41e3-8424-77c154d7382e/volumes"
Sep 06 01:12:33.366573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:33.366542    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:38.345258 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:38.345219    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"]
Sep 06 01:12:38.410767 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:38.410736    2575 scope.go:117] "RemoveContainer" containerID="2aadc277f1fdf42dd55d270974481758bd022227ff6806354f246e0e6bffd4ae"
Sep 06 01:12:38.419705 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:38.419685    2575 scope.go:117] "RemoveContainer" containerID="fad9d60758021778c1bf3e0ebe8927d3c3dd70f14ed9d0c294142d7e6c6970e7"
Sep 06 01:12:38.440312 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:38.440288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:47.323609 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:47.323572    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:12:53.366965 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.366926    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55" podUID="4b61fdb4-f612-4f19-a620-30c16a27d2db" containerName="main" containerID="cri-o://554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f" gracePeriod=45
Sep 06 01:12:53.619330 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.619277    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:12:53.654502 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.654469    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b61fdb4-f612-4f19-a620-30c16a27d2db" containerID="554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f" exitCode=0
Sep 06 01:12:53.654634 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.654513    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"
Sep 06 01:12:53.654634 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.654552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55" event={"ID":"4b61fdb4-f612-4f19-a620-30c16a27d2db","Type":"ContainerDied","Data":"554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f"}
Sep 06 01:12:53.654634 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.654601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55" event={"ID":"4b61fdb4-f612-4f19-a620-30c16a27d2db","Type":"ContainerDied","Data":"29c5a78f620f12d1a36500a709e0f1dc2b0af985d12824e6cd2b7f91896f7e46"}
Sep 06 01:12:53.654634 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.654625    2575 scope.go:117] "RemoveContainer" containerID="554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f"
Sep 06 01:12:53.664475 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.664455    2575 scope.go:117] "RemoveContainer" containerID="554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f"
Sep 06 01:12:53.664759 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:12:53.664738    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f\": container with ID starting with 554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f not found: ID does not exist" containerID="554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f"
Sep 06 01:12:53.664881 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.664765    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f"} err="failed to get container status \"554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f\": rpc error: code = NotFound desc = could not find container \"554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f\": container with ID starting with 554429e2184e546ebc35d7ae6d3c84c5bb6f211bd49f7ce6422759da28ec759f not found: ID does not exist"
Sep 06 01:12:53.696162 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696141    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-model-cache\") pod \"4b61fdb4-f612-4f19-a620-30c16a27d2db\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") "
Sep 06 01:12:53.696281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696170    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-dshm\") pod \"4b61fdb4-f612-4f19-a620-30c16a27d2db\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") "
Sep 06 01:12:53.696281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b61fdb4-f612-4f19-a620-30c16a27d2db-tls-certs\") pod \"4b61fdb4-f612-4f19-a620-30c16a27d2db\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") "
Sep 06 01:12:53.696356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696287    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj9hj\" (UniqueName: \"kubernetes.io/projected/4b61fdb4-f612-4f19-a620-30c16a27d2db-kube-api-access-vj9hj\") pod \"4b61fdb4-f612-4f19-a620-30c16a27d2db\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") "
Sep 06 01:12:53.696356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696313    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-tmp-dir\") pod \"4b61fdb4-f612-4f19-a620-30c16a27d2db\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") "
Sep 06 01:12:53.696356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696349    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-home\") pod \"4b61fdb4-f612-4f19-a620-30c16a27d2db\" (UID: \"4b61fdb4-f612-4f19-a620-30c16a27d2db\") "
Sep 06 01:12:53.696515 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696388    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b61fdb4-f612-4f19-a620-30c16a27d2db" (UID: "4b61fdb4-f612-4f19-a620-30c16a27d2db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:12:53.696638 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696615    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b61fdb4-f612-4f19-a620-30c16a27d2db" (UID: "4b61fdb4-f612-4f19-a620-30c16a27d2db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:12:53.696707 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696655    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-home" (OuterVolumeSpecName: "home") pod "4b61fdb4-f612-4f19-a620-30c16a27d2db" (UID: "4b61fdb4-f612-4f19-a620-30c16a27d2db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:12:53.696772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696755    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:53.696806 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696775    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:53.696806 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.696785    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:53.698630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.698606    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-dshm" (OuterVolumeSpecName: "dshm") pod "4b61fdb4-f612-4f19-a620-30c16a27d2db" (UID: "4b61fdb4-f612-4f19-a620-30c16a27d2db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:12:53.698630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.698610    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b61fdb4-f612-4f19-a620-30c16a27d2db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b61fdb4-f612-4f19-a620-30c16a27d2db" (UID: "4b61fdb4-f612-4f19-a620-30c16a27d2db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:12:53.698761 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.698647    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b61fdb4-f612-4f19-a620-30c16a27d2db-kube-api-access-vj9hj" (OuterVolumeSpecName: "kube-api-access-vj9hj") pod "4b61fdb4-f612-4f19-a620-30c16a27d2db" (UID: "4b61fdb4-f612-4f19-a620-30c16a27d2db"). InnerVolumeSpecName "kube-api-access-vj9hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:12:53.798238 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.798209    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b61fdb4-f612-4f19-a620-30c16a27d2db-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:53.798238 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.798232    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b61fdb4-f612-4f19-a620-30c16a27d2db-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:53.798238 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.798241    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj9hj\" (UniqueName: \"kubernetes.io/projected/4b61fdb4-f612-4f19-a620-30c16a27d2db-kube-api-access-vj9hj\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:53.979254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.979220    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"]
Sep 06 01:12:53.981904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:53.981879    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7534e03d/e2e-lifecycle-model-7534e03d-kserve-5c8d9899ff-gsv55"]
Sep 06 01:12:54.202643 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:54.202609    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b61fdb4-f612-4f19-a620-30c16a27d2db" path="/var/lib/kubelet/pods/4b61fdb4-f612-4f19-a620-30c16a27d2db/volumes"
Sep 06 01:12:55.997110 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997079    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7781-62c0d9"]
Sep 06 01:12:55.997486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997450    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3932d42d-3541-41e3-8424-77c154d7382e" containerName="payload-processing"
Sep 06 01:12:55.997486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997462    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3932d42d-3541-41e3-8424-77c154d7382e" containerName="payload-processing"
Sep 06 01:12:55.997486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997478    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b61fdb4-f612-4f19-a620-30c16a27d2db" containerName="main"
Sep 06 01:12:55.997486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997484    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b61fdb4-f612-4f19-a620-30c16a27d2db" containerName="main"
Sep 06 01:12:55.997630 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997534    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b61fdb4-f612-4f19-a620-30c16a27d2db" containerName="main"
Sep 06 01:12:55.997956 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:55.997944    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3932d42d-3541-41e3-8424-77c154d7382e" containerName="payload-processing"
Sep 06 01:12:56.002280 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.002262    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:56.004907 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.004882    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tmqsk\""
Sep 06 01:12:56.006388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.006366    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7781-62c0d9"]
Sep 06 01:12:56.123455 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.123422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjb6n\" (UniqueName: \"kubernetes.io/projected/3a740dff-6ae1-4c19-9c0f-68fc50d00993-kube-api-access-vjb6n\") pod \"mt-curl-7781-62c0d9\" (UID: \"3a740dff-6ae1-4c19-9c0f-68fc50d00993\") " pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:56.224118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.224082    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjb6n\" (UniqueName: \"kubernetes.io/projected/3a740dff-6ae1-4c19-9c0f-68fc50d00993-kube-api-access-vjb6n\") pod \"mt-curl-7781-62c0d9\" (UID: \"3a740dff-6ae1-4c19-9c0f-68fc50d00993\") " pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:56.232477 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.232445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjb6n\" (UniqueName: \"kubernetes.io/projected/3a740dff-6ae1-4c19-9c0f-68fc50d00993-kube-api-access-vjb6n\") pod \"mt-curl-7781-62c0d9\" (UID: \"3a740dff-6ae1-4c19-9c0f-68fc50d00993\") " pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:56.313652 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.313577    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:56.586183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.586157    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7781-62c0d9"]
Sep 06 01:12:56.588615 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:12:56.588585    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a740dff_6ae1_4c19_9c0f_68fc50d00993.slice/crio-281d339cb9ea920b8233e64d05bf8ebe3267f7e4613498494df523de12ef5444 WatchSource:0}: Error finding container 281d339cb9ea920b8233e64d05bf8ebe3267f7e4613498494df523de12ef5444: Status 404 returned error can't find the container with id 281d339cb9ea920b8233e64d05bf8ebe3267f7e4613498494df523de12ef5444
Sep 06 01:12:56.674892 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:56.674845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7781-62c0d9" event={"ID":"3a740dff-6ae1-4c19-9c0f-68fc50d00993","Type":"ContainerStarted","Data":"281d339cb9ea920b8233e64d05bf8ebe3267f7e4613498494df523de12ef5444"}
Sep 06 01:12:57.681968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:57.681874    2575 generic.go:356] "Generic (PLEG): container finished" podID="3a740dff-6ae1-4c19-9c0f-68fc50d00993" containerID="065fa3a6b33f5bd3a67f6a4073b89fa6b228072748f3987f48b55890a24acd9e" exitCode=0
Sep 06 01:12:57.682326 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:57.681967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7781-62c0d9" event={"ID":"3a740dff-6ae1-4c19-9c0f-68fc50d00993","Type":"ContainerDied","Data":"065fa3a6b33f5bd3a67f6a4073b89fa6b228072748f3987f48b55890a24acd9e"}
Sep 06 01:12:58.818936 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:58.818912    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:58.841624 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:58.841599    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7781-62c0d9_3a740dff-6ae1-4c19-9c0f-68fc50d00993/mt-curl-7781-62c0d9/0.log"
Sep 06 01:12:58.865148 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:58.865117    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7781-62c0d9"]
Sep 06 01:12:58.868555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:58.868534    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7781-62c0d9"]
Sep 06 01:12:58.948239 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:58.948216    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjb6n\" (UniqueName: \"kubernetes.io/projected/3a740dff-6ae1-4c19-9c0f-68fc50d00993-kube-api-access-vjb6n\") pod \"3a740dff-6ae1-4c19-9c0f-68fc50d00993\" (UID: \"3a740dff-6ae1-4c19-9c0f-68fc50d00993\") "
Sep 06 01:12:58.950403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:58.950382    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a740dff-6ae1-4c19-9c0f-68fc50d00993-kube-api-access-vjb6n" (OuterVolumeSpecName: "kube-api-access-vjb6n") pod "3a740dff-6ae1-4c19-9c0f-68fc50d00993" (UID: "3a740dff-6ae1-4c19-9c0f-68fc50d00993"). InnerVolumeSpecName "kube-api-access-vjb6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:12:59.049085 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:59.049058    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjb6n\" (UniqueName: \"kubernetes.io/projected/3a740dff-6ae1-4c19-9c0f-68fc50d00993-kube-api-access-vjb6n\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:12:59.695060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:59.695032    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="281d339cb9ea920b8233e64d05bf8ebe3267f7e4613498494df523de12ef5444"
Sep 06 01:12:59.695253 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:12:59.695062    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:12:59.704698 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:12:59.704673    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7781-62c0d9\" is forbidden: User \"system:node:ip-10-0-129-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-5.ec2.internal' and this object" podUID="3a740dff-6ae1-4c19-9c0f-68fc50d00993" pod="opendatahub/mt-curl-7781-62c0d9"
Sep 06 01:13:00.201828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:00.201793    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a740dff-6ae1-4c19-9c0f-68fc50d00993" path="/var/lib/kubelet/pods/3a740dff-6ae1-4c19-9c0f-68fc50d00993/volumes"
Sep 06 01:13:00.244585 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:00.244545    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"]
Sep 06 01:13:00.262091 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:00.262059    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:01.351475 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:01.351431    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"]
Sep 06 01:13:01.364547 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:01.364510    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:15.265376 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.265328    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf" podUID="8459b165-c3e8-4b24-9267-cb4e61d10eca" containerName="main" containerID="cri-o://a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6" gracePeriod=45
Sep 06 01:13:15.380304 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.380273    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:15.513323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.513295    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:13:15.596434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.596351    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:15.605138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-home\") pod \"8459b165-c3e8-4b24-9267-cb4e61d10eca\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") "
Sep 06 01:13:15.605281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605170    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8459b165-c3e8-4b24-9267-cb4e61d10eca-tls-certs\") pod \"8459b165-c3e8-4b24-9267-cb4e61d10eca\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") "
Sep 06 01:13:15.605281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605203    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-dshm\") pod \"8459b165-c3e8-4b24-9267-cb4e61d10eca\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") "
Sep 06 01:13:15.605281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605270    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdwcf\" (UniqueName: \"kubernetes.io/projected/8459b165-c3e8-4b24-9267-cb4e61d10eca-kube-api-access-rdwcf\") pod \"8459b165-c3e8-4b24-9267-cb4e61d10eca\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") "
Sep 06 01:13:15.605449 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605280    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-home" (OuterVolumeSpecName: "home") pod "8459b165-c3e8-4b24-9267-cb4e61d10eca" (UID: "8459b165-c3e8-4b24-9267-cb4e61d10eca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:15.605449 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605319    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-tmp-dir\") pod \"8459b165-c3e8-4b24-9267-cb4e61d10eca\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") "
Sep 06 01:13:15.605449 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605357    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-model-cache\") pod \"8459b165-c3e8-4b24-9267-cb4e61d10eca\" (UID: \"8459b165-c3e8-4b24-9267-cb4e61d10eca\") "
Sep 06 01:13:15.605597 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605562    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8459b165-c3e8-4b24-9267-cb4e61d10eca" (UID: "8459b165-c3e8-4b24-9267-cb4e61d10eca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:15.605647 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605617    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-model-cache" (OuterVolumeSpecName: "model-cache") pod "8459b165-c3e8-4b24-9267-cb4e61d10eca" (UID: "8459b165-c3e8-4b24-9267-cb4e61d10eca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:15.605696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605682    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:15.605739 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605699    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:15.605739 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.605709    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:15.607481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.607462    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8459b165-c3e8-4b24-9267-cb4e61d10eca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8459b165-c3e8-4b24-9267-cb4e61d10eca" (UID: "8459b165-c3e8-4b24-9267-cb4e61d10eca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:13:15.607890 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.607833    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-dshm" (OuterVolumeSpecName: "dshm") pod "8459b165-c3e8-4b24-9267-cb4e61d10eca" (UID: "8459b165-c3e8-4b24-9267-cb4e61d10eca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:15.607890 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.607840    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8459b165-c3e8-4b24-9267-cb4e61d10eca-kube-api-access-rdwcf" (OuterVolumeSpecName: "kube-api-access-rdwcf") pod "8459b165-c3e8-4b24-9267-cb4e61d10eca" (UID: "8459b165-c3e8-4b24-9267-cb4e61d10eca"). InnerVolumeSpecName "kube-api-access-rdwcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:13:15.706143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.706110    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8459b165-c3e8-4b24-9267-cb4e61d10eca-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:15.706143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.706137    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8459b165-c3e8-4b24-9267-cb4e61d10eca-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:15.706143 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.706146    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdwcf\" (UniqueName: \"kubernetes.io/projected/8459b165-c3e8-4b24-9267-cb4e61d10eca-kube-api-access-rdwcf\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:15.793734 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.793696    2575 generic.go:356] "Generic (PLEG): container finished" podID="8459b165-c3e8-4b24-9267-cb4e61d10eca" containerID="a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6" exitCode=0
Sep 06 01:13:15.793909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.793746    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"
Sep 06 01:13:15.793909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.793782    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf" event={"ID":"8459b165-c3e8-4b24-9267-cb4e61d10eca","Type":"ContainerDied","Data":"a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6"}
Sep 06 01:13:15.793909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.793832    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf" event={"ID":"8459b165-c3e8-4b24-9267-cb4e61d10eca","Type":"ContainerDied","Data":"6cd3be534b59779cc0952844cd5b653ebf1ee9088246619ae5e144080056960c"}
Sep 06 01:13:15.793909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.793870    2575 scope.go:117] "RemoveContainer" containerID="a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6"
Sep 06 01:13:15.803599 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.803585    2575 scope.go:117] "RemoveContainer" containerID="a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6"
Sep 06 01:13:15.803898 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:13:15.803872    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6\": container with ID starting with a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6 not found: ID does not exist" containerID="a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6"
Sep 06 01:13:15.804007 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.803908    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6"} err="failed to get container status \"a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6\": rpc error: code = NotFound desc = could not find container \"a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6\": container with ID starting with a158a20251565d5cb7b0896c161123dabb84334a284326c6a4d298162c16afc6 not found: ID does not exist"
Sep 06 01:13:15.815064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.815042    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"]
Sep 06 01:13:15.819495 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:15.819475    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/auth-test-model-5844a8-kserve-bb8db9c6c-ff7tf"]
Sep 06 01:13:16.203162 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.203129    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8459b165-c3e8-4b24-9267-cb4e61d10eca" path="/var/lib/kubelet/pods/8459b165-c3e8-4b24-9267-cb4e61d10eca/volumes"
Sep 06 01:13:16.372648 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.372609    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5" podUID="dc9f3434-3a40-40b0-b08f-780975591922" containerName="main" containerID="cri-o://53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355" gracePeriod=45
Sep 06 01:13:16.622136 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.622115    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:13:16.716495 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716465    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-home\") pod \"dc9f3434-3a40-40b0-b08f-780975591922\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") "
Sep 06 01:13:16.716658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716509    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-tmp-dir\") pod \"dc9f3434-3a40-40b0-b08f-780975591922\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") "
Sep 06 01:13:16.716658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716535    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc9f3434-3a40-40b0-b08f-780975591922-tls-certs\") pod \"dc9f3434-3a40-40b0-b08f-780975591922\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") "
Sep 06 01:13:16.716658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716589    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x645f\" (UniqueName: \"kubernetes.io/projected/dc9f3434-3a40-40b0-b08f-780975591922-kube-api-access-x645f\") pod \"dc9f3434-3a40-40b0-b08f-780975591922\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") "
Sep 06 01:13:16.716658 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716645    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-model-cache\") pod \"dc9f3434-3a40-40b0-b08f-780975591922\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") "
Sep 06 01:13:16.716894 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716694    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-dshm\") pod \"dc9f3434-3a40-40b0-b08f-780975591922\" (UID: \"dc9f3434-3a40-40b0-b08f-780975591922\") "
Sep 06 01:13:16.716894 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716751    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-home" (OuterVolumeSpecName: "home") pod "dc9f3434-3a40-40b0-b08f-780975591922" (UID: "dc9f3434-3a40-40b0-b08f-780975591922"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:16.716894 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716766    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc9f3434-3a40-40b0-b08f-780975591922" (UID: "dc9f3434-3a40-40b0-b08f-780975591922"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:16.717033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.716963    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc9f3434-3a40-40b0-b08f-780975591922" (UID: "dc9f3434-3a40-40b0-b08f-780975591922"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:16.717086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.717060    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:16.717086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.717072    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:16.717086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.717080    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:16.718988 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.718964    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9f3434-3a40-40b0-b08f-780975591922-kube-api-access-x645f" (OuterVolumeSpecName: "kube-api-access-x645f") pod "dc9f3434-3a40-40b0-b08f-780975591922" (UID: "dc9f3434-3a40-40b0-b08f-780975591922"). InnerVolumeSpecName "kube-api-access-x645f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:13:16.719085 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.718989    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9f3434-3a40-40b0-b08f-780975591922-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc9f3434-3a40-40b0-b08f-780975591922" (UID: "dc9f3434-3a40-40b0-b08f-780975591922"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:13:16.719085 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.718997    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-dshm" (OuterVolumeSpecName: "dshm") pod "dc9f3434-3a40-40b0-b08f-780975591922" (UID: "dc9f3434-3a40-40b0-b08f-780975591922"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:13:16.800737 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.800652    2575 generic.go:356] "Generic (PLEG): container finished" podID="dc9f3434-3a40-40b0-b08f-780975591922" containerID="53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355" exitCode=0
Sep 06 01:13:16.800737 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.800693    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"
Sep 06 01:13:16.800960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.800738    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5" event={"ID":"dc9f3434-3a40-40b0-b08f-780975591922","Type":"ContainerDied","Data":"53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355"}
Sep 06 01:13:16.800960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.800784    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5" event={"ID":"dc9f3434-3a40-40b0-b08f-780975591922","Type":"ContainerDied","Data":"591b98f91c743cbaa3d51a8b8c7068dc98112f3c7fb423333e6839ed8009b794"}
Sep 06 01:13:16.800960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.800807    2575 scope.go:117] "RemoveContainer" containerID="53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355"
Sep 06 01:13:16.810607 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.810587    2575 scope.go:117] "RemoveContainer" containerID="53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355"
Sep 06 01:13:16.810902 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:13:16.810842    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355\": container with ID starting with 53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355 not found: ID does not exist" containerID="53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355"
Sep 06 01:13:16.810971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.810895    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355"} err="failed to get container status \"53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355\": rpc error: code = NotFound desc = could not find container \"53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355\": container with ID starting with 53e936f6e8a3745d61e1a898e771774e40806ec9ced2337f1968f12e8bba4355 not found: ID does not exist"
Sep 06 01:13:16.817683 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.817662    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc9f3434-3a40-40b0-b08f-780975591922-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:16.817770 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.817684    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x645f\" (UniqueName: \"kubernetes.io/projected/dc9f3434-3a40-40b0-b08f-780975591922-kube-api-access-x645f\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:16.817770 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.817698    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc9f3434-3a40-40b0-b08f-780975591922-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:13:16.822536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.822514    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"]
Sep 06 01:13:16.828237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:16.828215    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/auth-test-model-470346-kserve-db44c4467-zm4k5"]
Sep 06 01:13:18.201564 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:18.201531    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9f3434-3a40-40b0-b08f-780975591922" path="/var/lib/kubelet/pods/dc9f3434-3a40-40b0-b08f-780975591922/volumes"
Sep 06 01:13:21.192625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.192565    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:21.271043 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271009    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"]
Sep 06 01:13:21.271372 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271359    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9f3434-3a40-40b0-b08f-780975591922" containerName="main"
Sep 06 01:13:21.271417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271374    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9f3434-3a40-40b0-b08f-780975591922" containerName="main"
Sep 06 01:13:21.271417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271383    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a740dff-6ae1-4c19-9c0f-68fc50d00993" containerName="mt-curl-7781-62c0d9"
Sep 06 01:13:21.271417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271390    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a740dff-6ae1-4c19-9c0f-68fc50d00993" containerName="mt-curl-7781-62c0d9"
Sep 06 01:13:21.271417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271400    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8459b165-c3e8-4b24-9267-cb4e61d10eca" containerName="main"
Sep 06 01:13:21.271417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271404    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8459b165-c3e8-4b24-9267-cb4e61d10eca" containerName="main"
Sep 06 01:13:21.271568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271472    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a740dff-6ae1-4c19-9c0f-68fc50d00993" containerName="mt-curl-7781-62c0d9"
Sep 06 01:13:21.271568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271480    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9f3434-3a40-40b0-b08f-780975591922" containerName="main"
Sep 06 01:13:21.271568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.271488    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8459b165-c3e8-4b24-9267-cb4e61d10eca" containerName="main"
Sep 06 01:13:21.275869 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.275836    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.278336 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.278313    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"kube-root-ca.crt\""
Sep 06 01:13:21.279096 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.279073    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"openshift-service-ca.crt\""
Sep 06 01:13:21.279227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.279100    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"sub-test-model-5844a8-kserve-self-signed-certs\""
Sep 06 01:13:21.279227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.279139    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"default-dockercfg-tnbc6\""
Sep 06 01:13:21.286582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.286557    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"]
Sep 06 01:13:21.307541 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.307517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:21.353313 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.353290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr78r\" (UniqueName: \"kubernetes.io/projected/fe22d4d1-1da3-4925-9b87-5e333ad284a2-kube-api-access-xr78r\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.353423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.353336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tmp-dir\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.353423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.353403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-dshm\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.353514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.353435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-home\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.353514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.353484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-model-cache\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.353514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.353504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tls-certs\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.454745 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.454717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-model-cache\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.454968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.454754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tls-certs\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.454968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.454781    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr78r\" (UniqueName: \"kubernetes.io/projected/fe22d4d1-1da3-4925-9b87-5e333ad284a2-kube-api-access-xr78r\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.454968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.454833    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tmp-dir\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.454968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.454908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-dshm\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.454968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.454942    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-home\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.455236 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.455171    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-model-cache\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.455236 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.455206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tmp-dir\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.455312 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.455301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-home\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.457232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.457210    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-dshm\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.457390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.457372    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tls-certs\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.463795 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.463770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr78r\" (UniqueName: \"kubernetes.io/projected/fe22d4d1-1da3-4925-9b87-5e333ad284a2-kube-api-access-xr78r\") pod \"sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.585281 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.585252    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:21.861133 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:21.861107    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"]
Sep 06 01:13:21.862593 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:13:21.862569    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe22d4d1_1da3_4925_9b87_5e333ad284a2.slice/crio-ef7d83ac9ec3d4f45cfe24a266eff4e7c73997780d24cdf0fe5fb89c4b5d3593 WatchSource:0}: Error finding container ef7d83ac9ec3d4f45cfe24a266eff4e7c73997780d24cdf0fe5fb89c4b5d3593: Status 404 returned error can't find the container with id ef7d83ac9ec3d4f45cfe24a266eff4e7c73997780d24cdf0fe5fb89c4b5d3593
Sep 06 01:13:22.843457 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:22.843422    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" event={"ID":"fe22d4d1-1da3-4925-9b87-5e333ad284a2","Type":"ContainerStarted","Data":"3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6"}
Sep 06 01:13:22.843457 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:22.843463    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" event={"ID":"fe22d4d1-1da3-4925-9b87-5e333ad284a2","Type":"ContainerStarted","Data":"ef7d83ac9ec3d4f45cfe24a266eff4e7c73997780d24cdf0fe5fb89c4b5d3593"}
Sep 06 01:13:22.860109 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:22.860063    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" podStartSLOduration=1.860046453 podStartE2EDuration="1.860046453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:13:22.859004739 +0000 UTC m=+1365.188363601" watchObservedRunningTime="2026-09-06 01:13:22.860046453 +0000 UTC m=+1365.189405294"
Sep 06 01:13:27.025136 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:27.025105    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:27.124702 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:27.124669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:27.290133 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:27.290027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:28.578694 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:28.578664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:28.681090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:28.681059    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:28.778256 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:28.778221    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:29.398580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:29.398537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:29.493313 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:29.493285    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:29.592884 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:29.592843    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:30.855378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:30.855340    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:31.586392 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:31.586357    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:31.586597 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:31.586417    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:31.598829 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:31.598806    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:31.912070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:31.911992    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:13:35.862322 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.862288    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"]
Sep 06 01:13:35.865772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.865751    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:35.868239 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.868220    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"kube-root-ca.crt\""
Sep 06 01:13:35.869071 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.869048    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"openshift-service-ca.crt\""
Sep 06 01:13:35.869153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.869076    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"sub-test-model-470346-kserve-self-signed-certs\""
Sep 06 01:13:35.869153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.869099    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-470346\"/\"default-dockercfg-pvm25\""
Sep 06 01:13:35.874537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.874499    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"]
Sep 06 01:13:35.888550 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.888521    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:35.986421 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.986388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvf4\" (UniqueName: \"kubernetes.io/projected/e2be7b6e-16fa-42be-954c-731058e0c53f-kube-api-access-9gvf4\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:35.986580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.986428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-dshm\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:35.986580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.986449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-home\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:35.986580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.986496    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2be7b6e-16fa-42be-954c-731058e0c53f-tls-certs\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:35.986580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.986562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-model-cache\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:35.986735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:35.986585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-tmp-dir\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087259 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-tmp-dir\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvf4\" (UniqueName: \"kubernetes.io/projected/e2be7b6e-16fa-42be-954c-731058e0c53f-kube-api-access-9gvf4\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087332    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-dshm\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-home\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2be7b6e-16fa-42be-954c-731058e0c53f-tls-certs\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087434 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087403    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-model-cache\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087716 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087679    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-tmp-dir\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087716 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-home\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.087944 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.087924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-model-cache\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.089746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.089726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-dshm\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.090025 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.090009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2be7b6e-16fa-42be-954c-731058e0c53f-tls-certs\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.094933 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.094907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvf4\" (UniqueName: \"kubernetes.io/projected/e2be7b6e-16fa-42be-954c-731058e0c53f-kube-api-access-9gvf4\") pod \"sub-test-model-470346-kserve-5d54b87fff-zzpbg\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") " pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.177078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.177000    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:36.448337 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.448311    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"]
Sep 06 01:13:36.449723 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:13:36.449693    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2be7b6e_16fa_42be_954c_731058e0c53f.slice/crio-a7403c69442fac31eda106fd5eb6e66c57c84dbd3ea58e57051a532469507d7d WatchSource:0}: Error finding container a7403c69442fac31eda106fd5eb6e66c57c84dbd3ea58e57051a532469507d7d: Status 404 returned error can't find the container with id a7403c69442fac31eda106fd5eb6e66c57c84dbd3ea58e57051a532469507d7d
Sep 06 01:13:36.934078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.934046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg" event={"ID":"e2be7b6e-16fa-42be-954c-731058e0c53f","Type":"ContainerStarted","Data":"ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f"}
Sep 06 01:13:36.934464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.934086    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg" event={"ID":"e2be7b6e-16fa-42be-954c-731058e0c53f","Type":"ContainerStarted","Data":"a7403c69442fac31eda106fd5eb6e66c57c84dbd3ea58e57051a532469507d7d"}
Sep 06 01:13:36.971216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:36.971165    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg" podStartSLOduration=1.9711452029999998 podStartE2EDuration="1.971145203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:13:36.968874299 +0000 UTC m=+1379.298233144" watchObservedRunningTime="2026-09-06 01:13:36.971145203 +0000 UTC m=+1379.300504039"
Sep 06 01:13:37.016522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:37.016490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:37.154383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:37.154354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:37.717522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:37.717490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:38.449631 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:38.449596    2575 scope.go:117] "RemoveContainer" containerID="13c10fbbd3e1cd23fd52c036d78b7342e0f299506bc2d9b1b56e1cfb97d5b2ed"
Sep 06 01:13:38.459128 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:38.459103    2575 scope.go:117] "RemoveContainer" containerID="5cc3df5edf5fe9a48e5151d8b9a6984e0d3a8fd680bb8998e4e1791f96df614d"
Sep 06 01:13:43.450922 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:43.450888    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:43.559393 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:43.559360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:43.651973 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:43.651943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:46.177511 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:46.177461    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:46.177511 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:46.177524    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:46.190077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:46.190052    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:47.003044 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:47.003015    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:13:54.161040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:54.161005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:13:54.398485 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:13:54.398439    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:15.473529 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:15.473475    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:16.876847 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:16.876813    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"]
Sep 06 01:14:16.895425 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:16.895354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:17.983943 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:17.983913    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"]
Sep 06 01:14:18.007866 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:18.007815    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:21.255700 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:21.255666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:21.322012 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:21.321979    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:21.625130 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:21.625045    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:21.934790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:21.934714    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:22.232396 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:22.232360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:24.874407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:24.874377    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:24.969599 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:24.969547    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:26.075108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.075070    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:26.135023 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.134986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:26.307029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.306996    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:26.649427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.649401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:26.747688 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.747655    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:26.798393 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.798356    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"]
Sep 06 01:14:26.802104 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.802083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:26.804166 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.804132    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-2d5f29\""
Sep 06 01:14:26.804278 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.804210    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-2d5f29-dockercfg-mqv79\""
Sep 06 01:14:26.812242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.812220    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"]
Sep 06 01:14:26.855795 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.855761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8tc\" (UniqueName: \"kubernetes.io/projected/002d0153-4b1a-42e7-95de-978ea73f4660-kube-api-access-nd8tc\") pod \"payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") " pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:26.856021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.855824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/002d0153-4b1a-42e7-95de-978ea73f4660-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") " pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:26.956690 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.956661    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8tc\" (UniqueName: \"kubernetes.io/projected/002d0153-4b1a-42e7-95de-978ea73f4660-kube-api-access-nd8tc\") pod \"payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") " pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:26.956848 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.956711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/002d0153-4b1a-42e7-95de-978ea73f4660-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") " pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:26.957384 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.957365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/002d0153-4b1a-42e7-95de-978ea73f4660-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") " pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:26.964959 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:26.964939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8tc\" (UniqueName: \"kubernetes.io/projected/002d0153-4b1a-42e7-95de-978ea73f4660-kube-api-access-nd8tc\") pod \"payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") " pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:27.113378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:27.113345    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:27.385157 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:27.385130    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"]
Sep 06 01:14:27.387397 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:14:27.387367    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002d0153_4b1a_42e7_95de_978ea73f4660.slice/crio-72ccde6d9581f2a1fa0f41d03369224011d753cc45470c929f1fde67889b021f WatchSource:0}: Error finding container 72ccde6d9581f2a1fa0f41d03369224011d753cc45470c929f1fde67889b021f: Status 404 returned error can't find the container with id 72ccde6d9581f2a1fa0f41d03369224011d753cc45470c929f1fde67889b021f
Sep 06 01:14:28.247076 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:28.247040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546" event={"ID":"002d0153-4b1a-42e7-95de-978ea73f4660","Type":"ContainerStarted","Data":"e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1"}
Sep 06 01:14:28.247449 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:28.247103    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:28.247449 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:28.247124    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546" event={"ID":"002d0153-4b1a-42e7-95de-978ea73f4660","Type":"ContainerStarted","Data":"72ccde6d9581f2a1fa0f41d03369224011d753cc45470c929f1fde67889b021f"}
Sep 06 01:14:28.265465 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:28.265414    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546" podStartSLOduration=2.265397616 podStartE2EDuration="2.265397616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:14:28.263599815 +0000 UTC m=+1430.592958686" watchObservedRunningTime="2026-09-06 01:14:28.265397616 +0000 UTC m=+1430.594756453"
Sep 06 01:14:31.897661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:31.897618    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" podUID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" containerName="main" containerID="cri-o://3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6" gracePeriod=45
Sep 06 01:14:31.901339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:31.901308    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" podUID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.92:8000/ready\": dial tcp 10.133.0.92:8000: connect: connection refused"
Sep 06 01:14:32.150344 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.150285    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:14:32.278480 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.278437    2575 generic.go:356] "Generic (PLEG): container finished" podID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" containerID="3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6" exitCode=0
Sep 06 01:14:32.278653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.278482    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" event={"ID":"fe22d4d1-1da3-4925-9b87-5e333ad284a2","Type":"ContainerDied","Data":"3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6"}
Sep 06 01:14:32.278653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.278501    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"
Sep 06 01:14:32.278653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.278528    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk" event={"ID":"fe22d4d1-1da3-4925-9b87-5e333ad284a2","Type":"ContainerDied","Data":"ef7d83ac9ec3d4f45cfe24a266eff4e7c73997780d24cdf0fe5fb89c4b5d3593"}
Sep 06 01:14:32.278653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.278544    2575 scope.go:117] "RemoveContainer" containerID="3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6"
Sep 06 01:14:32.287453 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.287437    2575 scope.go:117] "RemoveContainer" containerID="3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6"
Sep 06 01:14:32.287713 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:14:32.287696    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6\": container with ID starting with 3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6 not found: ID does not exist" containerID="3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6"
Sep 06 01:14:32.287763 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.287720    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6"} err="failed to get container status \"3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6\": rpc error: code = NotFound desc = could not find container \"3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6\": container with ID starting with 3c20a248d756d99b95783f791e625d8971862b3438e79ec100e03493ca84a2c6 not found: ID does not exist"
Sep 06 01:14:32.307042 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307023    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-home\") pod \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") "
Sep 06 01:14:32.307124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307059    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-dshm\") pod \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") "
Sep 06 01:14:32.307124 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307113    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-model-cache\") pod \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") "
Sep 06 01:14:32.307227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307168    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tls-certs\") pod \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") "
Sep 06 01:14:32.307227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307189    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr78r\" (UniqueName: \"kubernetes.io/projected/fe22d4d1-1da3-4925-9b87-5e333ad284a2-kube-api-access-xr78r\") pod \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") "
Sep 06 01:14:32.307227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307214    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tmp-dir\") pod \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\" (UID: \"fe22d4d1-1da3-4925-9b87-5e333ad284a2\") "
Sep 06 01:14:32.307379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307299    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-home" (OuterVolumeSpecName: "home") pod "fe22d4d1-1da3-4925-9b87-5e333ad284a2" (UID: "fe22d4d1-1da3-4925-9b87-5e333ad284a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:32.307431 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307374    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe22d4d1-1da3-4925-9b87-5e333ad284a2" (UID: "fe22d4d1-1da3-4925-9b87-5e333ad284a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:32.307527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307497    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe22d4d1-1da3-4925-9b87-5e333ad284a2" (UID: "fe22d4d1-1da3-4925-9b87-5e333ad284a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:32.307641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307547    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:32.307641 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.307565    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:32.309306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.309282    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-dshm" (OuterVolumeSpecName: "dshm") pod "fe22d4d1-1da3-4925-9b87-5e333ad284a2" (UID: "fe22d4d1-1da3-4925-9b87-5e333ad284a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:32.309403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.309359    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe22d4d1-1da3-4925-9b87-5e333ad284a2" (UID: "fe22d4d1-1da3-4925-9b87-5e333ad284a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:14:32.309459 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.309438    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe22d4d1-1da3-4925-9b87-5e333ad284a2-kube-api-access-xr78r" (OuterVolumeSpecName: "kube-api-access-xr78r") pod "fe22d4d1-1da3-4925-9b87-5e333ad284a2" (UID: "fe22d4d1-1da3-4925-9b87-5e333ad284a2"). InnerVolumeSpecName "kube-api-access-xr78r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:14:32.409021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.408959    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:32.409021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.408981    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr78r\" (UniqueName: \"kubernetes.io/projected/fe22d4d1-1da3-4925-9b87-5e333ad284a2-kube-api-access-xr78r\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:32.409021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.408990    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:32.409021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.408999    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe22d4d1-1da3-4925-9b87-5e333ad284a2-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:32.600040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.600011    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"]
Sep 06 01:14:32.605767 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:32.605742    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/sub-test-model-5844a8-kserve-7f5bfdd6f5-4tjlk"]
Sep 06 01:14:33.003467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.003423    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg" podUID="e2be7b6e-16fa-42be-954c-731058e0c53f" containerName="main" containerID="cri-o://ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f" gracePeriod=45
Sep 06 01:14:33.256153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.256092    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:14:33.286107 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.286073    2575 generic.go:356] "Generic (PLEG): container finished" podID="e2be7b6e-16fa-42be-954c-731058e0c53f" containerID="ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f" exitCode=0
Sep 06 01:14:33.286107 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.286107    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"
Sep 06 01:14:33.286296 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.286151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg" event={"ID":"e2be7b6e-16fa-42be-954c-731058e0c53f","Type":"ContainerDied","Data":"ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f"}
Sep 06 01:14:33.286296 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.286194    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg" event={"ID":"e2be7b6e-16fa-42be-954c-731058e0c53f","Type":"ContainerDied","Data":"a7403c69442fac31eda106fd5eb6e66c57c84dbd3ea58e57051a532469507d7d"}
Sep 06 01:14:33.286296 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.286216    2575 scope.go:117] "RemoveContainer" containerID="ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f"
Sep 06 01:14:33.295233 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.295220    2575 scope.go:117] "RemoveContainer" containerID="ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f"
Sep 06 01:14:33.295505 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:14:33.295488    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f\": container with ID starting with ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f not found: ID does not exist" containerID="ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f"
Sep 06 01:14:33.295611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.295513    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f"} err="failed to get container status \"ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f\": rpc error: code = NotFound desc = could not find container \"ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f\": container with ID starting with ba8938c1614b7a05181dd197438ab64c5bda6a06e5aabd710a357889e92f734f not found: ID does not exist"
Sep 06 01:14:33.418790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.418754    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-home\") pod \"e2be7b6e-16fa-42be-954c-731058e0c53f\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") "
Sep 06 01:14:33.419005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.418807    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-model-cache\") pod \"e2be7b6e-16fa-42be-954c-731058e0c53f\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") "
Sep 06 01:14:33.419005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.418892    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gvf4\" (UniqueName: \"kubernetes.io/projected/e2be7b6e-16fa-42be-954c-731058e0c53f-kube-api-access-9gvf4\") pod \"e2be7b6e-16fa-42be-954c-731058e0c53f\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") "
Sep 06 01:14:33.419005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.418928    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-dshm\") pod \"e2be7b6e-16fa-42be-954c-731058e0c53f\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") "
Sep 06 01:14:33.419005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.418956    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2be7b6e-16fa-42be-954c-731058e0c53f-tls-certs\") pod \"e2be7b6e-16fa-42be-954c-731058e0c53f\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") "
Sep 06 01:14:33.419242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419076    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-tmp-dir\") pod \"e2be7b6e-16fa-42be-954c-731058e0c53f\" (UID: \"e2be7b6e-16fa-42be-954c-731058e0c53f\") "
Sep 06 01:14:33.419242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419088    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-home" (OuterVolumeSpecName: "home") pod "e2be7b6e-16fa-42be-954c-731058e0c53f" (UID: "e2be7b6e-16fa-42be-954c-731058e0c53f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:33.419242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419147    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-model-cache" (OuterVolumeSpecName: "model-cache") pod "e2be7b6e-16fa-42be-954c-731058e0c53f" (UID: "e2be7b6e-16fa-42be-954c-731058e0c53f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:33.419407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419335    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e2be7b6e-16fa-42be-954c-731058e0c53f" (UID: "e2be7b6e-16fa-42be-954c-731058e0c53f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:33.419463 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419440    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:33.419463 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419455    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:33.419574 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.419464    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:33.421185 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.421155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-dshm" (OuterVolumeSpecName: "dshm") pod "e2be7b6e-16fa-42be-954c-731058e0c53f" (UID: "e2be7b6e-16fa-42be-954c-731058e0c53f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:14:33.421690 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.421671    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2be7b6e-16fa-42be-954c-731058e0c53f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e2be7b6e-16fa-42be-954c-731058e0c53f" (UID: "e2be7b6e-16fa-42be-954c-731058e0c53f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:14:33.421746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.421670    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2be7b6e-16fa-42be-954c-731058e0c53f-kube-api-access-9gvf4" (OuterVolumeSpecName: "kube-api-access-9gvf4") pod "e2be7b6e-16fa-42be-954c-731058e0c53f" (UID: "e2be7b6e-16fa-42be-954c-731058e0c53f"). InnerVolumeSpecName "kube-api-access-9gvf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:14:33.520060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.519989    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gvf4\" (UniqueName: \"kubernetes.io/projected/e2be7b6e-16fa-42be-954c-731058e0c53f-kube-api-access-9gvf4\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:33.520060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.520013    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2be7b6e-16fa-42be-954c-731058e0c53f-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:33.520060 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.520022    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2be7b6e-16fa-42be-954c-731058e0c53f-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:14:33.609085 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.609050    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"]
Sep 06 01:14:33.614693 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:33.614663    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-470346/sub-test-model-470346-kserve-5d54b87fff-zzpbg"]
Sep 06 01:14:34.202804 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.202770    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2be7b6e-16fa-42be-954c-731058e0c53f" path="/var/lib/kubelet/pods/e2be7b6e-16fa-42be-954c-731058e0c53f/volumes"
Sep 06 01:14:34.203215 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.203199    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" path="/var/lib/kubelet/pods/fe22d4d1-1da3-4925-9b87-5e333ad284a2/volumes"
Sep 06 01:14:34.595653 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.595567    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"]
Sep 06 01:14:34.596072 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.596053    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" containerName="main"
Sep 06 01:14:34.596072 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.596073    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" containerName="main"
Sep 06 01:14:34.596237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.596097    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2be7b6e-16fa-42be-954c-731058e0c53f" containerName="main"
Sep 06 01:14:34.596237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.596104    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2be7b6e-16fa-42be-954c-731058e0c53f" containerName="main"
Sep 06 01:14:34.596237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.596206    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe22d4d1-1da3-4925-9b87-5e333ad284a2" containerName="main"
Sep 06 01:14:34.596237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.596219    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2be7b6e-16fa-42be-954c-731058e0c53f" containerName="main"
Sep 06 01:14:34.600681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.600662    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.603000 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.602982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"openshift-service-ca.crt\""
Sep 06 01:14:34.603705 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.603689    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"default-dockercfg-tnbc6\""
Sep 06 01:14:34.603786 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.603706    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"kube-root-ca.crt\""
Sep 06 01:14:34.603786 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.603689    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5844a8\"/\"rate-test-model-5844a8-kserve-self-signed-certs\""
Sep 06 01:14:34.609340 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.609171    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"]
Sep 06 01:14:34.692082 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.692050    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:34.730223 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.730192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-dshm\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.730358 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.730225    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda73724-07ef-47b0-83a6-9c51a78c57ec-tls-certs\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.730358 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.730269    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-tmp-dir\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.730358 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.730316    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-home\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.730358 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.730350    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-model-cache\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.730496 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.730455    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpr2x\" (UniqueName: \"kubernetes.io/projected/fda73724-07ef-47b0-83a6-9c51a78c57ec-kube-api-access-rpr2x\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.831560 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.831526    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-tmp-dir\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.831560 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.831565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-home\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.831814 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.831594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-model-cache\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.831814 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.831645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpr2x\" (UniqueName: \"kubernetes.io/projected/fda73724-07ef-47b0-83a6-9c51a78c57ec-kube-api-access-rpr2x\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.831814 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.831665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-dshm\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.831814 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.831681    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda73724-07ef-47b0-83a6-9c51a78c57ec-tls-certs\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.832070 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.832043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-tmp-dir\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.832114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.832050    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-home\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.832114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.832100    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-model-cache\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.834153 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.834131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-dshm\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.834406 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.834388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda73724-07ef-47b0-83a6-9c51a78c57ec-tls-certs\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.839476 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.839450    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpr2x\" (UniqueName: \"kubernetes.io/projected/fda73724-07ef-47b0-83a6-9c51a78c57ec-kube-api-access-rpr2x\") pod \"rate-test-model-5844a8-kserve-666464dbc-vrwkb\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") " pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:34.911580 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:34.911514    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:35.186247 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:35.186214    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"]
Sep 06 01:14:35.186929 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:14:35.186905    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfda73724_07ef_47b0_83a6_9c51a78c57ec.slice/crio-79175541add2f8f9b7dbfb2fe1b74d031692d9bca7aec69b5612d70ae2cb900f WatchSource:0}: Error finding container 79175541add2f8f9b7dbfb2fe1b74d031692d9bca7aec69b5612d70ae2cb900f: Status 404 returned error can't find the container with id 79175541add2f8f9b7dbfb2fe1b74d031692d9bca7aec69b5612d70ae2cb900f
Sep 06 01:14:35.302028 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:35.301990    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb" event={"ID":"fda73724-07ef-47b0-83a6-9c51a78c57ec","Type":"ContainerStarted","Data":"806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5"}
Sep 06 01:14:35.302028 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:35.302033    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb" event={"ID":"fda73724-07ef-47b0-83a6-9c51a78c57ec","Type":"ContainerStarted","Data":"79175541add2f8f9b7dbfb2fe1b74d031692d9bca7aec69b5612d70ae2cb900f"}
Sep 06 01:14:35.324871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:35.324802    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb" podStartSLOduration=1.324784704 podStartE2EDuration="1.324784704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:14:35.322145872 +0000 UTC m=+1437.651504709" watchObservedRunningTime="2026-09-06 01:14:35.324784704 +0000 UTC m=+1437.654143539"
Sep 06 01:14:38.498934 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:38.498900    2575 scope.go:117] "RemoveContainer" containerID="fd0fa6d965fd5c2ba900c41811afe88571f2288753de6e1b8e7bdd7cc68bcf98"
Sep 06 01:14:39.255550 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:39.255523    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:14:44.912383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:44.912343    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:44.912772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:44.912399    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:44.925035 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:44.925008    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:45.368880 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:45.368838    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:14:48.947229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:48.947190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:49.926130 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:49.926098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:50.396993 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.396960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:50.698583 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.698557    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"]
Sep 06 01:14:50.703767 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.703741    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.705923 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.705897    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1f7961-openshift-default-dockercfg-5qdg8\""
Sep 06 01:14:50.714961 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.714936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"]
Sep 06 01:14:50.730036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.730016    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:50.778051 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778026    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-credential-socket\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-envoy\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88t5p\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-kube-api-access-88t5p\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778226 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778144    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-certs\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778226 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istiod-ca-cert\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778291 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-data\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778291 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-socket\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778357 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778297    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-token\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.778391 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.778353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-podinfo\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.879699 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istiod-ca-cert\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.879897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-data\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.879897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879814    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-socket\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-token\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879942    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-podinfo\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-credential-socket\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.879983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-envoy\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880266 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880126    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-data\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880266 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880174    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88t5p\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-kube-api-access-88t5p\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880266 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-certs\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880441    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-socket\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880497    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istiod-ca-cert\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880605 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-certs\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.880605 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.880590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-credential-socket\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.882252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.882230    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-envoy\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.882567 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.882548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-podinfo\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.888142 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.888123    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88t5p\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-kube-api-access-88t5p\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.888249 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.888138    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-token\") pod \"e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") " pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:50.980639 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:50.980564    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:51.015897 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.015870    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:51.294795 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.294766    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"]
Sep 06 01:14:51.295666 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:14:51.295639    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f1672f_d865_4b7f_959c_2b9bf8618d27.slice/crio-019edb2f83d41c1a89aca4f5da91fea5f7deefaf1123d4cf81b73789790ffb54 WatchSource:0}: Error finding container 019edb2f83d41c1a89aca4f5da91fea5f7deefaf1123d4cf81b73789790ffb54: Status 404 returned error can't find the container with id 019edb2f83d41c1a89aca4f5da91fea5f7deefaf1123d4cf81b73789790ffb54
Sep 06 01:14:51.298032 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.297995    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:14:51.298114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.298086    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:14:51.298154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.298134    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:14:51.395433 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.395387    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" event={"ID":"d4f1672f-d865-4b7f-959c-2b9bf8618d27","Type":"ContainerStarted","Data":"9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"}
Sep 06 01:14:51.395637 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.395492    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" event={"ID":"d4f1672f-d865-4b7f-959c-2b9bf8618d27","Type":"ContainerStarted","Data":"019edb2f83d41c1a89aca4f5da91fea5f7deefaf1123d4cf81b73789790ffb54"}
Sep 06 01:14:51.415796 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:51.415747    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podStartSLOduration=1.415728615 podStartE2EDuration="1.415728615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:14:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:14:51.413107095 +0000 UTC m=+1453.742465950" watchObservedRunningTime="2026-09-06 01:14:51.415728615 +0000 UTC m=+1453.745087450"
Sep 06 01:14:52.017012 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:52.016980    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:52.022017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:52.021993    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:52.401110 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:52.401030    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:52.402016 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:52.401997    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:14:53.405272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:53.405228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:53.498343 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:53.498310    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:54.553174 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:54.553146    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:54.650056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:54.650022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:55.113364 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.113324    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:55.193604 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.193564    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:55.242551 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.242517    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"]
Sep 06 01:14:55.245998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.245983    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.248379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.248358    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1f7961-dockercfg-ggzct\""
Sep 06 01:14:55.248493 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.248366    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1f7961\""
Sep 06 01:14:55.255466 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.255445    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"]
Sep 06 01:14:55.257068 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.257035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:14:55.264034 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.264012    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"]
Sep 06 01:14:55.268163 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.268147    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.278086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.278065    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"]
Sep 06 01:14:55.322301 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.322274    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20468152-2014-44d5-a8fd-2bd94a93e709-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.322423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.322307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqcsq\" (UniqueName: \"kubernetes.io/projected/724dec89-f00c-4c82-ae9a-0f6993f8e610-kube-api-access-bqcsq\") pod \"payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.322423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.322331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724dec89-f00c-4c82-ae9a-0f6993f8e610-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.322423 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.322368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrzjf\" (UniqueName: \"kubernetes.io/projected/20468152-2014-44d5-a8fd-2bd94a93e709-kube-api-access-vrzjf\") pod \"payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.423789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.423706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20468152-2014-44d5-a8fd-2bd94a93e709-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.423789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.423744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqcsq\" (UniqueName: \"kubernetes.io/projected/724dec89-f00c-4c82-ae9a-0f6993f8e610-kube-api-access-bqcsq\") pod \"payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.423789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.423770    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724dec89-f00c-4c82-ae9a-0f6993f8e610-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.423789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.423787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrzjf\" (UniqueName: \"kubernetes.io/projected/20468152-2014-44d5-a8fd-2bd94a93e709-kube-api-access-vrzjf\") pod \"payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.424506 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.424487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20468152-2014-44d5-a8fd-2bd94a93e709-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.424558 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.424529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724dec89-f00c-4c82-ae9a-0f6993f8e610-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.432047 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.432017    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqcsq\" (UniqueName: \"kubernetes.io/projected/724dec89-f00c-4c82-ae9a-0f6993f8e610-kube-api-access-bqcsq\") pod \"payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.432047 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.432017    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrzjf\" (UniqueName: \"kubernetes.io/projected/20468152-2014-44d5-a8fd-2bd94a93e709-kube-api-access-vrzjf\") pod \"payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") " pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.557953 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.557898    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:55.578780 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.578747    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:55.850084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.850056    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"]
Sep 06 01:14:55.852211 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:14:55.852186    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724dec89_f00c_4c82_ae9a_0f6993f8e610.slice/crio-8595c8f0033ac437180fbd5d57bc79dbc47acf4d4347f9f9b53262769e78a026 WatchSource:0}: Error finding container 8595c8f0033ac437180fbd5d57bc79dbc47acf4d4347f9f9b53262769e78a026: Status 404 returned error can't find the container with id 8595c8f0033ac437180fbd5d57bc79dbc47acf4d4347f9f9b53262769e78a026
Sep 06 01:14:55.881231 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:55.881207    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"]
Sep 06 01:14:55.885047 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:14:55.885018    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20468152_2014_44d5_a8fd_2bd94a93e709.slice/crio-2a9d16980ac9390baaeee48622298a2d17f8303a24fbe018a9c9eda2cebd028f WatchSource:0}: Error finding container 2a9d16980ac9390baaeee48622298a2d17f8303a24fbe018a9c9eda2cebd028f: Status 404 returned error can't find the container with id 2a9d16980ac9390baaeee48622298a2d17f8303a24fbe018a9c9eda2cebd028f
Sep 06 01:14:56.426206 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.426150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv" event={"ID":"724dec89-f00c-4c82-ae9a-0f6993f8e610","Type":"ContainerStarted","Data":"49707d9786c21980a729f958cb335870bb2001f6d37dddf071ddf04334f90d2e"}
Sep 06 01:14:56.426413 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.426327    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:14:56.426413 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.426358    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv" event={"ID":"724dec89-f00c-4c82-ae9a-0f6993f8e610","Type":"ContainerStarted","Data":"8595c8f0033ac437180fbd5d57bc79dbc47acf4d4347f9f9b53262769e78a026"}
Sep 06 01:14:56.427845 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.427819    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c" event={"ID":"20468152-2014-44d5-a8fd-2bd94a93e709","Type":"ContainerStarted","Data":"486764e021d461894872e6512c142016cb66c09e4ab6ec0b38dc33bb8b02f22a"}
Sep 06 01:14:56.427995 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.427891    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:14:56.427995 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.427907    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c" event={"ID":"20468152-2014-44d5-a8fd-2bd94a93e709","Type":"ContainerStarted","Data":"2a9d16980ac9390baaeee48622298a2d17f8303a24fbe018a9c9eda2cebd028f"}
Sep 06 01:14:56.444321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.444273    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv" podStartSLOduration=1.44425564 podStartE2EDuration="1.44425564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:14:56.441394069 +0000 UTC m=+1458.770752928" watchObservedRunningTime="2026-09-06 01:14:56.44425564 +0000 UTC m=+1458.773614479"
Sep 06 01:14:56.457490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:14:56.457445    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c" podStartSLOduration=1.457433443 podStartE2EDuration="1.457433443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:14:56.455928961 +0000 UTC m=+1458.785287820" watchObservedRunningTime="2026-09-06 01:14:56.457433443 +0000 UTC m=+1458.786792279"
Sep 06 01:15:00.150966 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.150928    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"]
Sep 06 01:15:00.155221 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.155202    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"]
Sep 06 01:15:00.155344 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.155330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"
Sep 06 01:15:00.157695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.157678    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:15:00.158557 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.158538    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"
Sep 06 01:15:00.170439 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.169929    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"]
Sep 06 01:15:00.171134 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.170998    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"
Sep 06 01:15:00.173771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.173745    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"]
Sep 06 01:15:00.179694 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.179667    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"]
Sep 06 01:15:00.180128 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.180109    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"
Sep 06 01:15:00.184531 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.184512    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"
Sep 06 01:15:00.207396 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.207371    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"]
Sep 06 01:15:00.228470 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.228382    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"
Sep 06 01:15:00.465025 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.464984    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"]
Sep 06 01:15:00.466402 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:15:00.466367    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b759535_2923_416b_a424_f22262735bde.slice/crio-5228c3016dec0f96eb4f49f83c9a13cdd8adad2b27bf102aabf1074fa99e63b6 WatchSource:0}: Error finding container 5228c3016dec0f96eb4f49f83c9a13cdd8adad2b27bf102aabf1074fa99e63b6: Status 404 returned error can't find the container with id 5228c3016dec0f96eb4f49f83c9a13cdd8adad2b27bf102aabf1074fa99e63b6
Sep 06 01:15:00.482127 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.479576    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"]
Sep 06 01:15:00.482868 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:15:00.482593    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa0aec3_8023_4e86_b399_80cd19e2f9fe.slice/crio-0a84011fce320fe1f3bf94ce71f2daaf2d5c759715c97639ea72771eac4c090c WatchSource:0}: Error finding container 0a84011fce320fe1f3bf94ce71f2daaf2d5c759715c97639ea72771eac4c090c: Status 404 returned error can't find the container with id 0a84011fce320fe1f3bf94ce71f2daaf2d5c759715c97639ea72771eac4c090c
Sep 06 01:15:00.531793 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:00.531766    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"]
Sep 06 01:15:00.534689 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:15:00.534666    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3296540b_0624_4d40_a4c9_c89469eca25e.slice/crio-bc062de7052ddf4e211db375f09f68fa37a933b45ef7fe2d0c22ca8553598d7c WatchSource:0}: Error finding container bc062de7052ddf4e211db375f09f68fa37a933b45ef7fe2d0c22ca8553598d7c: Status 404 returned error can't find the container with id bc062de7052ddf4e211db375f09f68fa37a933b45ef7fe2d0c22ca8553598d7c
Sep 06 01:15:01.460390 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.460350    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm" event={"ID":"aaa0aec3-8023-4e86-b399-80cd19e2f9fe","Type":"ContainerStarted","Data":"59317a87a7abd2c9bced5d509b193f7a46fe23c050d11896c0bef5469185cc49"}
Sep 06 01:15:01.460808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.460449    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm" event={"ID":"aaa0aec3-8023-4e86-b399-80cd19e2f9fe","Type":"ContainerStarted","Data":"0a84011fce320fe1f3bf94ce71f2daaf2d5c759715c97639ea72771eac4c090c"}
Sep 06 01:15:01.462182 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.462156    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk" event={"ID":"3296540b-0624-4d40-a4c9-c89469eca25e","Type":"ContainerStarted","Data":"f0be12b9c3dd214709dafb5d51da16026d923644df99334b475428c8754c6d45"}
Sep 06 01:15:01.462308 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.462202    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk" event={"ID":"3296540b-0624-4d40-a4c9-c89469eca25e","Type":"ContainerStarted","Data":"bc062de7052ddf4e211db375f09f68fa37a933b45ef7fe2d0c22ca8553598d7c"}
Sep 06 01:15:01.464021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.463988    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8" event={"ID":"1b759535-2923-416b-a424-f22262735bde","Type":"ContainerStarted","Data":"0a4a2bb268c6897c09abf9cff96b21ff48261d27448814387ae2233bdf7ca913"}
Sep 06 01:15:01.464106 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.464035    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8" event={"ID":"1b759535-2923-416b-a424-f22262735bde","Type":"ContainerStarted","Data":"5228c3016dec0f96eb4f49f83c9a13cdd8adad2b27bf102aabf1074fa99e63b6"}
Sep 06 01:15:01.477382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.477338    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm" podStartSLOduration=1.477318285 podStartE2EDuration="1.477318285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:15:01.475682995 +0000 UTC m=+1463.805041830" watchObservedRunningTime="2026-09-06 01:15:01.477318285 +0000 UTC m=+1463.806677120"
Sep 06 01:15:01.491714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.491661    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk" podStartSLOduration=1.4916463119999999 podStartE2EDuration="1.491646312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:15:01.489781006 +0000 UTC m=+1463.819139867" watchObservedRunningTime="2026-09-06 01:15:01.491646312 +0000 UTC m=+1463.821005193"
Sep 06 01:15:01.506214 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:01.506170    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8" podStartSLOduration=1.506155584 podStartE2EDuration="1.506155584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:15:01.505322541 +0000 UTC m=+1463.834681397" watchObservedRunningTime="2026-09-06 01:15:01.506155584 +0000 UTC m=+1463.835514419"
Sep 06 01:15:02.472413 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:02.472377    2575 generic.go:356] "Generic (PLEG): container finished" podID="1b759535-2923-416b-a424-f22262735bde" containerID="0a4a2bb268c6897c09abf9cff96b21ff48261d27448814387ae2233bdf7ca913" exitCode=0
Sep 06 01:15:02.472884 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:02.472425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8" event={"ID":"1b759535-2923-416b-a424-f22262735bde","Type":"ContainerDied","Data":"0a4a2bb268c6897c09abf9cff96b21ff48261d27448814387ae2233bdf7ca913"}
Sep 06 01:15:02.474207 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:02.474179    2575 generic.go:356] "Generic (PLEG): container finished" podID="aaa0aec3-8023-4e86-b399-80cd19e2f9fe" containerID="59317a87a7abd2c9bced5d509b193f7a46fe23c050d11896c0bef5469185cc49" exitCode=0
Sep 06 01:15:02.474324 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:02.474258    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm" event={"ID":"aaa0aec3-8023-4e86-b399-80cd19e2f9fe","Type":"ContainerDied","Data":"59317a87a7abd2c9bced5d509b193f7a46fe23c050d11896c0bef5469185cc49"}
Sep 06 01:15:02.475665 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:02.475643    2575 generic.go:356] "Generic (PLEG): container finished" podID="3296540b-0624-4d40-a4c9-c89469eca25e" containerID="f0be12b9c3dd214709dafb5d51da16026d923644df99334b475428c8754c6d45" exitCode=0
Sep 06 01:15:02.475746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:02.475713    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk" event={"ID":"3296540b-0624-4d40-a4c9-c89469eca25e","Type":"ContainerDied","Data":"f0be12b9c3dd214709dafb5d51da16026d923644df99334b475428c8754c6d45"}
Sep 06 01:15:03.646295 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:03.646269    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"
Sep 06 01:15:03.721369 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:03.721340    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"
Sep 06 01:15:03.740998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:03.740934    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"
Sep 06 01:15:04.492262 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.492177    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"
Sep 06 01:15:04.492262 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.492205    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm" event={"ID":"aaa0aec3-8023-4e86-b399-80cd19e2f9fe","Type":"ContainerDied","Data":"0a84011fce320fe1f3bf94ce71f2daaf2d5c759715c97639ea72771eac4c090c"}
Sep 06 01:15:04.492262 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.492256    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a84011fce320fe1f3bf94ce71f2daaf2d5c759715c97639ea72771eac4c090c"
Sep 06 01:15:04.493700 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.493683    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"
Sep 06 01:15:04.493828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.493682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk" event={"ID":"3296540b-0624-4d40-a4c9-c89469eca25e","Type":"ContainerDied","Data":"bc062de7052ddf4e211db375f09f68fa37a933b45ef7fe2d0c22ca8553598d7c"}
Sep 06 01:15:04.493828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.493792    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc062de7052ddf4e211db375f09f68fa37a933b45ef7fe2d0c22ca8553598d7c"
Sep 06 01:15:04.495312 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.495287    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8" event={"ID":"1b759535-2923-416b-a424-f22262735bde","Type":"ContainerDied","Data":"5228c3016dec0f96eb4f49f83c9a13cdd8adad2b27bf102aabf1074fa99e63b6"}
Sep 06 01:15:04.495312 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.495309    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29810955-rftq8"
Sep 06 01:15:04.495468 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.495331    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5228c3016dec0f96eb4f49f83c9a13cdd8adad2b27bf102aabf1074fa99e63b6"
Sep 06 01:15:04.867489 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:04.867399    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:07.435841 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:07.435806    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:15:07.436233 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:07.436047    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:15:10.323235 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:10.323194    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:10.384501 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:10.384470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:12.547444 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:12.547412    2575 generic.go:356] "Generic (PLEG): container finished" podID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerID="9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7" exitCode=0
Sep 06 01:15:12.547998 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:12.547491    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" event={"ID":"d4f1672f-d865-4b7f-959c-2b9bf8618d27","Type":"ContainerDied","Data":"9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"}
Sep 06 01:15:12.548084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:12.548065    2575 scope.go:117] "RemoveContainer" containerID="9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"
Sep 06 01:15:12.549258 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:12.549234    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:15:12.549324 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:12.549301    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:15:12.549360 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:12.549333    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:15:13.557020 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:13.556986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" event={"ID":"d4f1672f-d865-4b7f-959c-2b9bf8618d27","Type":"ContainerStarted","Data":"775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be"}
Sep 06 01:15:14.016271 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:14.016238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:15:14.016482 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:14.016458    2575 patch_prober.go:28] interesting pod/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 06 01:15:14.016538 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:14.016519    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 06 01:15:15.016697 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:15.016653    2575 patch_prober.go:28] interesting pod/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 06 01:15:15.017197 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:15.016714    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 06 01:15:16.017039 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:16.016999    2575 patch_prober.go:28] interesting pod/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 06 01:15:16.017441 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:16.017064    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 06 01:15:17.016773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:17.016731    2575 patch_prober.go:28] interesting pod/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 06 01:15:17.017011 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:17.016792    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 06 01:15:18.016536 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:18.016501    2575 patch_prober.go:28] interesting pod/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Sep 06 01:15:18.016939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:18.016563    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Sep 06 01:15:19.021415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:19.021386    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:15:19.021899 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:19.021884    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:15:19.022949 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:19.022928    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:15:19.206278 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:19.206246    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:19.264099 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:19.264070    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:21.365202 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:21.365172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:22.128935 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:22.128898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:22.366236 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:22.366201    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:22.464149 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:22.464114    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:37.195789 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.195756    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"]
Sep 06 01:15:37.201005 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.200976    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2d5f29-29810955-w2knk"]
Sep 06 01:15:37.203802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.203771    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:37.224442 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.224417    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"]
Sep 06 01:15:37.224681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.224658    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546" podUID="002d0153-4b1a-42e7-95de-978ea73f4660" containerName="payload-processing" containerID="cri-o://e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1" gracePeriod=30
Sep 06 01:15:37.277318 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.277284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:37.376340 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.376306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:37.476804 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.476776    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:37.479615 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.479591    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:15:37.539722 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.539696    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/002d0153-4b1a-42e7-95de-978ea73f4660-plugins-config-volume\") pod \"002d0153-4b1a-42e7-95de-978ea73f4660\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") "
Sep 06 01:15:37.539888 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.539790    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd8tc\" (UniqueName: \"kubernetes.io/projected/002d0153-4b1a-42e7-95de-978ea73f4660-kube-api-access-nd8tc\") pod \"002d0153-4b1a-42e7-95de-978ea73f4660\" (UID: \"002d0153-4b1a-42e7-95de-978ea73f4660\") "
Sep 06 01:15:37.540199 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.540173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/002d0153-4b1a-42e7-95de-978ea73f4660-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "002d0153-4b1a-42e7-95de-978ea73f4660" (UID: "002d0153-4b1a-42e7-95de-978ea73f4660"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:15:37.541883 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.541839    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/002d0153-4b1a-42e7-95de-978ea73f4660-kube-api-access-nd8tc" (OuterVolumeSpecName: "kube-api-access-nd8tc") pod "002d0153-4b1a-42e7-95de-978ea73f4660" (UID: "002d0153-4b1a-42e7-95de-978ea73f4660"). InnerVolumeSpecName "kube-api-access-nd8tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:15:37.641527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.641483    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd8tc\" (UniqueName: \"kubernetes.io/projected/002d0153-4b1a-42e7-95de-978ea73f4660-kube-api-access-nd8tc\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:37.641527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.641519    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/002d0153-4b1a-42e7-95de-978ea73f4660-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:37.717272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.717171    2575 generic.go:356] "Generic (PLEG): container finished" podID="002d0153-4b1a-42e7-95de-978ea73f4660" containerID="e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1" exitCode=0
Sep 06 01:15:37.717272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.717219    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"
Sep 06 01:15:37.717272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.717264    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546" event={"ID":"002d0153-4b1a-42e7-95de-978ea73f4660","Type":"ContainerDied","Data":"e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1"}
Sep 06 01:15:37.717521 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.717315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546" event={"ID":"002d0153-4b1a-42e7-95de-978ea73f4660","Type":"ContainerDied","Data":"72ccde6d9581f2a1fa0f41d03369224011d753cc45470c929f1fde67889b021f"}
Sep 06 01:15:37.717521 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.717336    2575 scope.go:117] "RemoveContainer" containerID="e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1"
Sep 06 01:15:37.740766 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.740747    2575 scope.go:117] "RemoveContainer" containerID="e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1"
Sep 06 01:15:37.741097 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:15:37.741076    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1\": container with ID starting with e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1 not found: ID does not exist" containerID="e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1"
Sep 06 01:15:37.741208 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.741105    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1"} err="failed to get container status \"e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1\": rpc error: code = NotFound desc = could not find container \"e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1\": container with ID starting with e72350dcf8f18ca7ee4e6c9e309119f6993039cea11b1c524754faa3bca4a9c1 not found: ID does not exist"
Sep 06 01:15:37.749784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.749755    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"]
Sep 06 01:15:37.754482 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.754460    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2d5f29-bdc775b47-2f546"]
Sep 06 01:15:37.762847 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:37.762823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:38.202351 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.202324    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="002d0153-4b1a-42e7-95de-978ea73f4660" path="/var/lib/kubelet/pods/002d0153-4b1a-42e7-95de-978ea73f4660/volumes"
Sep 06 01:15:38.202696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.202616    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3296540b-0624-4d40-a4c9-c89469eca25e" path="/var/lib/kubelet/pods/3296540b-0624-4d40-a4c9-c89469eca25e/volumes"
Sep 06 01:15:38.285178 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.285144    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:15:38.290378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.290361    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:15:38.528813 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.528728    2575 scope.go:117] "RemoveContainer" containerID="68ec4db4c9d4bdc58c32474dcb0b96f0d8db1f4cade0fec64ae472ac4c4b0322"
Sep 06 01:15:38.538048 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.538024    2575 scope.go:117] "RemoveContainer" containerID="c68349c6e4bc3baead8ed11330947cf9366334d86d213b1d4cf2da0b5c404053"
Sep 06 01:15:38.546036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:38.546017    2575 scope.go:117] "RemoveContainer" containerID="29221454570a502ec32ae20371407be539c14c4f27827baf8838d5cb37bd605d"
Sep 06 01:15:44.037373 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:44.037340    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:44.113800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:44.113771    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"]
Sep 06 01:15:44.116424 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:44.116385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:44.187378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:44.187348    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:44.315272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:44.315200    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:44.857084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:44.857040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:45.464740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:45.464702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:46.011330 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:46.011298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:46.314202 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:46.314125    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:46.441753 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:46.441726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:49.504874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:49.504830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:49.601652 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:49.601616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:50.092065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:50.092031    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:50.840210 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:50.840179    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:50.941934 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:50.941901    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:51.041288 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.041252    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:51.509341 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.509305    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:51.611362 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.611329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:51.638696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.638662    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"]
Sep 06 01:15:51.639352 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639331    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3296540b-0624-4d40-a4c9-c89469eca25e" containerName="cleanup"
Sep 06 01:15:51.639417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639358    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3296540b-0624-4d40-a4c9-c89469eca25e" containerName="cleanup"
Sep 06 01:15:51.639417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639389    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b759535-2923-416b-a424-f22262735bde" containerName="cleanup"
Sep 06 01:15:51.639417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639398    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b759535-2923-416b-a424-f22262735bde" containerName="cleanup"
Sep 06 01:15:51.639417 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639411    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa0aec3-8023-4e86-b399-80cd19e2f9fe" containerName="cleanup"
Sep 06 01:15:51.639537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639420    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa0aec3-8023-4e86-b399-80cd19e2f9fe" containerName="cleanup"
Sep 06 01:15:51.639537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639439    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="002d0153-4b1a-42e7-95de-978ea73f4660" containerName="payload-processing"
Sep 06 01:15:51.639537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639447    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="002d0153-4b1a-42e7-95de-978ea73f4660" containerName="payload-processing"
Sep 06 01:15:51.639636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639549    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="002d0153-4b1a-42e7-95de-978ea73f4660" containerName="payload-processing"
Sep 06 01:15:51.639636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639562    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3296540b-0624-4d40-a4c9-c89469eca25e" containerName="cleanup"
Sep 06 01:15:51.639636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639573    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa0aec3-8023-4e86-b399-80cd19e2f9fe" containerName="cleanup"
Sep 06 01:15:51.639636 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.639580    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b759535-2923-416b-a424-f22262735bde" containerName="cleanup"
Sep 06 01:15:51.645064 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.645042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.647379 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.647356    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b2e4f6-dockercfg-rsqxk\""
Sep 06 01:15:51.647487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.647356    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b2e4f6\""
Sep 06 01:15:51.652244 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.652213    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"]
Sep 06 01:15:51.782054 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.781955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40dfedf-864c-41cb-8a72-5ad7de2908fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.782190 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.782096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllpd\" (UniqueName: \"kubernetes.io/projected/e40dfedf-864c-41cb-8a72-5ad7de2908fb-kube-api-access-tllpd\") pod \"payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.883742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.883689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tllpd\" (UniqueName: \"kubernetes.io/projected/e40dfedf-864c-41cb-8a72-5ad7de2908fb-kube-api-access-tllpd\") pod \"payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.884245 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.883826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40dfedf-864c-41cb-8a72-5ad7de2908fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.884754 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.884725    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40dfedf-864c-41cb-8a72-5ad7de2908fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.892108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.892082    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllpd\" (UniqueName: \"kubernetes.io/projected/e40dfedf-864c-41cb-8a72-5ad7de2908fb-kube-api-access-tllpd\") pod \"payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:51.956977 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:51.956942    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:52.234089 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:52.234059    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"]
Sep 06 01:15:52.235097 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:15:52.235070    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40dfedf_864c_41cb_8a72_5ad7de2908fb.slice/crio-6acbf7f1d19f78d3193437f8fe71523ac468ebf6e301a39eb61737790226ac74 WatchSource:0}: Error finding container 6acbf7f1d19f78d3193437f8fe71523ac468ebf6e301a39eb61737790226ac74: Status 404 returned error can't find the container with id 6acbf7f1d19f78d3193437f8fe71523ac468ebf6e301a39eb61737790226ac74
Sep 06 01:15:52.812495 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:52.812461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c" event={"ID":"e40dfedf-864c-41cb-8a72-5ad7de2908fb","Type":"ContainerStarted","Data":"a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355"}
Sep 06 01:15:52.812673 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:52.812510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c" event={"ID":"e40dfedf-864c-41cb-8a72-5ad7de2908fb","Type":"ContainerStarted","Data":"6acbf7f1d19f78d3193437f8fe71523ac468ebf6e301a39eb61737790226ac74"}
Sep 06 01:15:52.812673 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:52.812541    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:15:52.830700 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:52.830652    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c" podStartSLOduration=1.830638939 podStartE2EDuration="1.830638939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:15:52.828071059 +0000 UTC m=+1515.157429917" watchObservedRunningTime="2026-09-06 01:15:52.830638939 +0000 UTC m=+1515.159997781"
Sep 06 01:15:55.012672 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:55.012635    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:15:59.134923 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.134817    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb" podUID="fda73724-07ef-47b0-83a6-9c51a78c57ec" containerName="main" containerID="cri-o://806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5" gracePeriod=45
Sep 06 01:15:59.389078 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.389016    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:15:59.555431 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555402    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-tmp-dir\") pod \"fda73724-07ef-47b0-83a6-9c51a78c57ec\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") "
Sep 06 01:15:59.555431 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555436    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-model-cache\") pod \"fda73724-07ef-47b0-83a6-9c51a78c57ec\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") "
Sep 06 01:15:59.555649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555488    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-home\") pod \"fda73724-07ef-47b0-83a6-9c51a78c57ec\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") "
Sep 06 01:15:59.555649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555507    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpr2x\" (UniqueName: \"kubernetes.io/projected/fda73724-07ef-47b0-83a6-9c51a78c57ec-kube-api-access-rpr2x\") pod \"fda73724-07ef-47b0-83a6-9c51a78c57ec\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") "
Sep 06 01:15:59.555649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555580    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda73724-07ef-47b0-83a6-9c51a78c57ec-tls-certs\") pod \"fda73724-07ef-47b0-83a6-9c51a78c57ec\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") "
Sep 06 01:15:59.555649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555612    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-dshm\") pod \"fda73724-07ef-47b0-83a6-9c51a78c57ec\" (UID: \"fda73724-07ef-47b0-83a6-9c51a78c57ec\") "
Sep 06 01:15:59.555809 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555653    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fda73724-07ef-47b0-83a6-9c51a78c57ec" (UID: "fda73724-07ef-47b0-83a6-9c51a78c57ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:15:59.555809 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555766    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "fda73724-07ef-47b0-83a6-9c51a78c57ec" (UID: "fda73724-07ef-47b0-83a6-9c51a78c57ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:15:59.555809 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555779    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-home" (OuterVolumeSpecName: "home") pod "fda73724-07ef-47b0-83a6-9c51a78c57ec" (UID: "fda73724-07ef-47b0-83a6-9c51a78c57ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:15:59.556004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555943    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:59.556004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555957    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:59.556004 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.555966    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:59.557871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.557832    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-dshm" (OuterVolumeSpecName: "dshm") pod "fda73724-07ef-47b0-83a6-9c51a78c57ec" (UID: "fda73724-07ef-47b0-83a6-9c51a78c57ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:15:59.557971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.557876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fda73724-07ef-47b0-83a6-9c51a78c57ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fda73724-07ef-47b0-83a6-9c51a78c57ec" (UID: "fda73724-07ef-47b0-83a6-9c51a78c57ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:15:59.557971 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.557921    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda73724-07ef-47b0-83a6-9c51a78c57ec-kube-api-access-rpr2x" (OuterVolumeSpecName: "kube-api-access-rpr2x") pod "fda73724-07ef-47b0-83a6-9c51a78c57ec" (UID: "fda73724-07ef-47b0-83a6-9c51a78c57ec"). InnerVolumeSpecName "kube-api-access-rpr2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:15:59.657337 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.657273    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda73724-07ef-47b0-83a6-9c51a78c57ec-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:59.657337 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.657296    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpr2x\" (UniqueName: \"kubernetes.io/projected/fda73724-07ef-47b0-83a6-9c51a78c57ec-kube-api-access-rpr2x\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:59.657337 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.657307    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda73724-07ef-47b0-83a6-9c51a78c57ec-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:15:59.854836 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.854799    2575 generic.go:356] "Generic (PLEG): container finished" podID="fda73724-07ef-47b0-83a6-9c51a78c57ec" containerID="806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5" exitCode=0
Sep 06 01:15:59.855029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.854842    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"
Sep 06 01:15:59.855029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.854890    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb" event={"ID":"fda73724-07ef-47b0-83a6-9c51a78c57ec","Type":"ContainerDied","Data":"806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5"}
Sep 06 01:15:59.855029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.854939    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb" event={"ID":"fda73724-07ef-47b0-83a6-9c51a78c57ec","Type":"ContainerDied","Data":"79175541add2f8f9b7dbfb2fe1b74d031692d9bca7aec69b5612d70ae2cb900f"}
Sep 06 01:15:59.855029 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.854958    2575 scope.go:117] "RemoveContainer" containerID="806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5"
Sep 06 01:15:59.864772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.864755    2575 scope.go:117] "RemoveContainer" containerID="806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5"
Sep 06 01:15:59.865037 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:15:59.865020    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5\": container with ID starting with 806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5 not found: ID does not exist" containerID="806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5"
Sep 06 01:15:59.865116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.865043    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5"} err="failed to get container status \"806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5\": rpc error: code = NotFound desc = could not find container \"806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5\": container with ID starting with 806cd11fd89f775426bc3ddb44e8e43c82b2c967fc3fe519da82656b508081c5 not found: ID does not exist"
Sep 06 01:15:59.875468 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.875446    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"]
Sep 06 01:15:59.879874 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:15:59.879837    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5844a8/rate-test-model-5844a8-kserve-666464dbc-vrwkb"]
Sep 06 01:16:00.202697 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.202668    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fda73724-07ef-47b0-83a6-9c51a78c57ec" path="/var/lib/kubelet/pods/fda73724-07ef-47b0-83a6-9c51a78c57ec/volumes"
Sep 06 01:16:00.605197 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.605119    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:00.641346 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.641313    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"]
Sep 06 01:16:00.641603 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.641574    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c" podUID="20468152-2014-44d5-a8fd-2bd94a93e709" containerName="payload-processing" containerID="cri-o://486764e021d461894872e6512c142016cb66c09e4ab6ec0b38dc33bb8b02f22a" gracePeriod=30
Sep 06 01:16:00.650911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.650884    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"]
Sep 06 01:16:00.651096 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.651075    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv" podUID="724dec89-f00c-4c82-ae9a-0f6993f8e610" containerName="payload-pre-processing" containerID="cri-o://49707d9786c21980a729f958cb335870bb2001f6d37dddf071ddf04334f90d2e" gracePeriod=30
Sep 06 01:16:00.684413 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.684387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:00.750654 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.750628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:00.821464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.821389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:00.866968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.866890    2575 generic.go:356] "Generic (PLEG): container finished" podID="724dec89-f00c-4c82-ae9a-0f6993f8e610" containerID="49707d9786c21980a729f958cb335870bb2001f6d37dddf071ddf04334f90d2e" exitCode=0
Sep 06 01:16:00.866968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.866946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv" event={"ID":"724dec89-f00c-4c82-ae9a-0f6993f8e610","Type":"ContainerDied","Data":"49707d9786c21980a729f958cb335870bb2001f6d37dddf071ddf04334f90d2e"}
Sep 06 01:16:00.868746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.868722    2575 generic.go:356] "Generic (PLEG): container finished" podID="20468152-2014-44d5-a8fd-2bd94a93e709" containerID="486764e021d461894872e6512c142016cb66c09e4ab6ec0b38dc33bb8b02f22a" exitCode=0
Sep 06 01:16:00.868903 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.868778    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c" event={"ID":"20468152-2014-44d5-a8fd-2bd94a93e709","Type":"ContainerDied","Data":"486764e021d461894872e6512c142016cb66c09e4ab6ec0b38dc33bb8b02f22a"}
Sep 06 01:16:00.894359 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.894327    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:00.915562 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.915538    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:16:00.939753 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:00.939727    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:16:01.073997 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.073958    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqcsq\" (UniqueName: \"kubernetes.io/projected/724dec89-f00c-4c82-ae9a-0f6993f8e610-kube-api-access-bqcsq\") pod \"724dec89-f00c-4c82-ae9a-0f6993f8e610\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") "
Sep 06 01:16:01.073997 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.074001    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20468152-2014-44d5-a8fd-2bd94a93e709-plugins-config-volume\") pod \"20468152-2014-44d5-a8fd-2bd94a93e709\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") "
Sep 06 01:16:01.074240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.074018    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrzjf\" (UniqueName: \"kubernetes.io/projected/20468152-2014-44d5-a8fd-2bd94a93e709-kube-api-access-vrzjf\") pod \"20468152-2014-44d5-a8fd-2bd94a93e709\" (UID: \"20468152-2014-44d5-a8fd-2bd94a93e709\") "
Sep 06 01:16:01.074240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.074082    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724dec89-f00c-4c82-ae9a-0f6993f8e610-plugins-config-volume\") pod \"724dec89-f00c-4c82-ae9a-0f6993f8e610\" (UID: \"724dec89-f00c-4c82-ae9a-0f6993f8e610\") "
Sep 06 01:16:01.074380 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.074352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20468152-2014-44d5-a8fd-2bd94a93e709-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20468152-2014-44d5-a8fd-2bd94a93e709" (UID: "20468152-2014-44d5-a8fd-2bd94a93e709"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:01.074477 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.074454    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/724dec89-f00c-4c82-ae9a-0f6993f8e610-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "724dec89-f00c-4c82-ae9a-0f6993f8e610" (UID: "724dec89-f00c-4c82-ae9a-0f6993f8e610"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:01.074526 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.074508    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20468152-2014-44d5-a8fd-2bd94a93e709-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:01.076835 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.076818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/724dec89-f00c-4c82-ae9a-0f6993f8e610-kube-api-access-bqcsq" (OuterVolumeSpecName: "kube-api-access-bqcsq") pod "724dec89-f00c-4c82-ae9a-0f6993f8e610" (UID: "724dec89-f00c-4c82-ae9a-0f6993f8e610"). InnerVolumeSpecName "kube-api-access-bqcsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:01.076907 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.076844    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20468152-2014-44d5-a8fd-2bd94a93e709-kube-api-access-vrzjf" (OuterVolumeSpecName: "kube-api-access-vrzjf") pod "20468152-2014-44d5-a8fd-2bd94a93e709" (UID: "20468152-2014-44d5-a8fd-2bd94a93e709"). InnerVolumeSpecName "kube-api-access-vrzjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:01.175710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.175630    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqcsq\" (UniqueName: \"kubernetes.io/projected/724dec89-f00c-4c82-ae9a-0f6993f8e610-kube-api-access-bqcsq\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:01.175710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.175657    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrzjf\" (UniqueName: \"kubernetes.io/projected/20468152-2014-44d5-a8fd-2bd94a93e709-kube-api-access-vrzjf\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:01.175710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.175666    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724dec89-f00c-4c82-ae9a-0f6993f8e610-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:01.227871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.227826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:01.876765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.876736    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"
Sep 06 01:16:01.876982 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.876777    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv" event={"ID":"724dec89-f00c-4c82-ae9a-0f6993f8e610","Type":"ContainerDied","Data":"8595c8f0033ac437180fbd5d57bc79dbc47acf4d4347f9f9b53262769e78a026"}
Sep 06 01:16:01.876982 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.876847    2575 scope.go:117] "RemoveContainer" containerID="49707d9786c21980a729f958cb335870bb2001f6d37dddf071ddf04334f90d2e"
Sep 06 01:16:01.878577 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.878563    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"
Sep 06 01:16:01.878647 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.878608    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c" event={"ID":"20468152-2014-44d5-a8fd-2bd94a93e709","Type":"ContainerDied","Data":"2a9d16980ac9390baaeee48622298a2d17f8303a24fbe018a9c9eda2cebd028f"}
Sep 06 01:16:01.887021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.887001    2575 scope.go:117] "RemoveContainer" containerID="486764e021d461894872e6512c142016cb66c09e4ab6ec0b38dc33bb8b02f22a"
Sep 06 01:16:01.900696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.900673    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"]
Sep 06 01:16:01.905591 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.905571    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1f7961-658f9858-668pv"]
Sep 06 01:16:01.919740 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.919671    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"]
Sep 06 01:16:01.923191 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:01.923169    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1f7961-78c96b5d77-mqm7c"]
Sep 06 01:16:02.201929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:02.201893    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20468152-2014-44d5-a8fd-2bd94a93e709" path="/var/lib/kubelet/pods/20468152-2014-44d5-a8fd-2bd94a93e709/volumes"
Sep 06 01:16:02.202282 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:02.202268    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="724dec89-f00c-4c82-ae9a-0f6993f8e610" path="/var/lib/kubelet/pods/724dec89-f00c-4c82-ae9a-0f6993f8e610/volumes"
Sep 06 01:16:03.821730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:03.821699    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:16:06.986272 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:06.986238    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:07.083215 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.083183    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:07.248725 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.248639    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"]
Sep 06 01:16:07.248922 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.248834    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy" containerID="cri-o://775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be" gracePeriod=30
Sep 06 01:16:07.272378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.272344    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:07.620206 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620130    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"]
Sep 06 01:16:07.620545 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620532    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20468152-2014-44d5-a8fd-2bd94a93e709" containerName="payload-processing"
Sep 06 01:16:07.620588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620546    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="20468152-2014-44d5-a8fd-2bd94a93e709" containerName="payload-processing"
Sep 06 01:16:07.620588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620562    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fda73724-07ef-47b0-83a6-9c51a78c57ec" containerName="main"
Sep 06 01:16:07.620588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620567    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fda73724-07ef-47b0-83a6-9c51a78c57ec" containerName="main"
Sep 06 01:16:07.620588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620582    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="724dec89-f00c-4c82-ae9a-0f6993f8e610" containerName="payload-pre-processing"
Sep 06 01:16:07.620588 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620588    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="724dec89-f00c-4c82-ae9a-0f6993f8e610" containerName="payload-pre-processing"
Sep 06 01:16:07.620793 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620670    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="20468152-2014-44d5-a8fd-2bd94a93e709" containerName="payload-processing"
Sep 06 01:16:07.620793 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620685    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fda73724-07ef-47b0-83a6-9c51a78c57ec" containerName="main"
Sep 06 01:16:07.620793 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.620694    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="724dec89-f00c-4c82-ae9a-0f6993f8e610" containerName="payload-pre-processing"
Sep 06 01:16:07.623735 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.623720    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:07.625951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.625936    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:16:07.626002 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.625937    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:16:07.631038 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.631002    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"]
Sep 06 01:16:07.723553 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.723523    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:07.736160 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.736134    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e566481c-3bee-447e-8715-dbebcdfc7b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx\" (UID: \"e566481c-3bee-447e-8715-dbebcdfc7b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:07.837240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.837194    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e566481c-3bee-447e-8715-dbebcdfc7b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx\" (UID: \"e566481c-3bee-447e-8715-dbebcdfc7b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:07.838036 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.838010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e566481c-3bee-447e-8715-dbebcdfc7b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx\" (UID: \"e566481c-3bee-447e-8715-dbebcdfc7b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:07.935802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:07.935718    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:08.208945 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:08.208918    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"]
Sep 06 01:16:08.210180 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:16:08.210160    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode566481c_3bee_447e_8715_dbebcdfc7b63.slice/crio-49c1cbe76152d2dc8c953665ec490f7d841fb5fce6f112101350f13e0d81de22 WatchSource:0}: Error finding container 49c1cbe76152d2dc8c953665ec490f7d841fb5fce6f112101350f13e0d81de22: Status 404 returned error can't find the container with id 49c1cbe76152d2dc8c953665ec490f7d841fb5fce6f112101350f13e0d81de22
Sep 06 01:16:08.925389 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:08.925300    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx" event={"ID":"e566481c-3bee-447e-8715-dbebcdfc7b63","Type":"ContainerStarted","Data":"14911a69f4f8d61fb855dabffb4f00acceb1cdbfbeaedeb6bacbe9b74f53faf1"}
Sep 06 01:16:08.925389 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:08.925357    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx" event={"ID":"e566481c-3bee-447e-8715-dbebcdfc7b63","Type":"ContainerStarted","Data":"49c1cbe76152d2dc8c953665ec490f7d841fb5fce6f112101350f13e0d81de22"}
Sep 06 01:16:08.941779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:08.941737    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx" podStartSLOduration=1.941724116 podStartE2EDuration="1.941724116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:16:08.939324184 +0000 UTC m=+1531.268683043" watchObservedRunningTime="2026-09-06 01:16:08.941724116 +0000 UTC m=+1531.271082952"
Sep 06 01:16:09.932104 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:09.932065    2575 generic.go:356] "Generic (PLEG): container finished" podID="e566481c-3bee-447e-8715-dbebcdfc7b63" containerID="14911a69f4f8d61fb855dabffb4f00acceb1cdbfbeaedeb6bacbe9b74f53faf1" exitCode=0
Sep 06 01:16:09.932514 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:09.932150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx" event={"ID":"e566481c-3bee-447e-8715-dbebcdfc7b63","Type":"ContainerDied","Data":"14911a69f4f8d61fb855dabffb4f00acceb1cdbfbeaedeb6bacbe9b74f53faf1"}
Sep 06 01:16:11.070447 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.070422    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:11.166756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.166727    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e566481c-3bee-447e-8715-dbebcdfc7b63-maas-api-service-ca\") pod \"e566481c-3bee-447e-8715-dbebcdfc7b63\" (UID: \"e566481c-3bee-447e-8715-dbebcdfc7b63\") "
Sep 06 01:16:11.167172 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.167145    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e566481c-3bee-447e-8715-dbebcdfc7b63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e566481c-3bee-447e-8715-dbebcdfc7b63" (UID: "e566481c-3bee-447e-8715-dbebcdfc7b63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:11.267809 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.267771    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e566481c-3bee-447e-8715-dbebcdfc7b63-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:11.883396 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.883360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:11.945930 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.945767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx" event={"ID":"e566481c-3bee-447e-8715-dbebcdfc7b63","Type":"ContainerDied","Data":"49c1cbe76152d2dc8c953665ec490f7d841fb5fce6f112101350f13e0d81de22"}
Sep 06 01:16:11.945930 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.945819    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49c1cbe76152d2dc8c953665ec490f7d841fb5fce6f112101350f13e0d81de22"
Sep 06 01:16:11.945930 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:11.945773    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"
Sep 06 01:16:12.515629 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.515570    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:16:12.579034 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579007    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-certs\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579054    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istiod-ca-cert\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579111    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-podinfo\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579145    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-credential-socket\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579167    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-envoy\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579193    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-data\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579211    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-socket\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579237    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-token\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579270    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88t5p\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-kube-api-access-88t5p\") pod \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\" (UID: \"d4f1672f-d865-4b7f-959c-2b9bf8618d27\") "
Sep 06 01:16:12.579546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579309    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:12.579546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579517    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:12.579546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579527    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:12.579546 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579539    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:12.579761 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579543    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.579761 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.579576    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:12.581656 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.581632    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-kube-api-access-88t5p" (OuterVolumeSpecName: "kube-api-access-88t5p") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "kube-api-access-88t5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:12.581777 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.581656    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:12.581986 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.581952    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:16:12.581986 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.581958    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4f1672f-d865-4b7f-959c-2b9bf8618d27" (UID: "d4f1672f-d865-4b7f-959c-2b9bf8618d27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:12.680627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680594    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680625    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680635    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680643    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680651    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f1672f-d865-4b7f-959c-2b9bf8618d27-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680659    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680667    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88t5p\" (UniqueName: \"kubernetes.io/projected/d4f1672f-d865-4b7f-959c-2b9bf8618d27-kube-api-access-88t5p\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.680800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.680676    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f1672f-d865-4b7f-959c-2b9bf8618d27-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:12.957089 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.957057    2575 generic.go:356] "Generic (PLEG): container finished" podID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerID="775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be" exitCode=0
Sep 06 01:16:12.957287 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.957104    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"
Sep 06 01:16:12.957287 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.957140    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" event={"ID":"d4f1672f-d865-4b7f-959c-2b9bf8618d27","Type":"ContainerDied","Data":"775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be"}
Sep 06 01:16:12.957287 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.957205    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz" event={"ID":"d4f1672f-d865-4b7f-959c-2b9bf8618d27","Type":"ContainerDied","Data":"019edb2f83d41c1a89aca4f5da91fea5f7deefaf1123d4cf81b73789790ffb54"}
Sep 06 01:16:12.957287 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.957235    2575 scope.go:117] "RemoveContainer" containerID="775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be"
Sep 06 01:16:12.967336 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.967321    2575 scope.go:117] "RemoveContainer" containerID="9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"
Sep 06 01:16:12.975689 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.975670    2575 scope.go:117] "RemoveContainer" containerID="775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be"
Sep 06 01:16:12.976102 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:12.976049    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be\": container with ID starting with 775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be not found: ID does not exist" containerID="775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be"
Sep 06 01:16:12.976241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.976160    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be"} err="failed to get container status \"775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be\": rpc error: code = NotFound desc = could not find container \"775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be\": container with ID starting with 775b22ea42d5aa39e6c23db99f2ab932e7357350add8b0e20349557cae5631be not found: ID does not exist"
Sep 06 01:16:12.976241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.976184    2575 scope.go:117] "RemoveContainer" containerID="9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"
Sep 06 01:16:12.976566 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:12.976543    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7\": container with ID starting with 9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7 not found: ID does not exist" containerID="9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"
Sep 06 01:16:12.976723 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.976686    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7"} err="failed to get container status \"9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7\": rpc error: code = NotFound desc = could not find container \"9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7\": container with ID starting with 9edf3c7ccacbeb72d26a0f403e62e9efd0bd49ba54a8a743cf2f14acac5f87a7 not found: ID does not exist"
Sep 06 01:16:12.979410 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.979391    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"]
Sep 06 01:16:12.983252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:12.983233    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1f7961-openshift-default-579dc5594b-nwfgz"]
Sep 06 01:16:13.888138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:13.888104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:14.202204 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:14.202171    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" path="/var/lib/kubelet/pods/d4f1672f-d865-4b7f-959c-2b9bf8618d27/volumes"
Sep 06 01:16:14.517058 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:14.516967    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:15.206219 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.206169    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:15.495837 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.495752    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"]
Sep 06 01:16:15.496233 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496216    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e566481c-3bee-447e-8715-dbebcdfc7b63" containerName="revoke-keys"
Sep 06 01:16:15.496335 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496236    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e566481c-3bee-447e-8715-dbebcdfc7b63" containerName="revoke-keys"
Sep 06 01:16:15.496335 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496258    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy"
Sep 06 01:16:15.496335 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496265    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy"
Sep 06 01:16:15.496335 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496275    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy"
Sep 06 01:16:15.496335 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496283    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy"
Sep 06 01:16:15.496586 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496381    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e566481c-3bee-447e-8715-dbebcdfc7b63" containerName="revoke-keys"
Sep 06 01:16:15.496586 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496394    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy"
Sep 06 01:16:15.496586 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.496403    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f1672f-d865-4b7f-959c-2b9bf8618d27" containerName="istio-proxy"
Sep 06 01:16:15.500985 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.500965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.503198 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.503176    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-074d50-openshift-default-dockercfg-6n542\""
Sep 06 01:16:15.510594 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.510569    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"]
Sep 06 01:16:15.520819 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.520789    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:15.606649 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606613    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d7jr\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-kube-api-access-5d7jr\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.606889 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-podinfo\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.606889 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606754    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-envoy\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.606889 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-certs\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.606889 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-token\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.607131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-socket\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.607131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.606961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e7f7f-834a-402c-9d96-295ac56f2d20-istiod-ca-cert\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.607131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.607017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-credential-socket\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.607131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.607056    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-data\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707515 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707488    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-token\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707536    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-socket\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e7f7f-834a-402c-9d96-295ac56f2d20-istiod-ca-cert\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-credential-socket\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707612    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-data\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d7jr\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-kube-api-access-5d7jr\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.707691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-podinfo\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707704    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-envoy\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.707733    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-certs\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708150 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.708050    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-credential-socket\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708150 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.708048    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-socket\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708150 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.708106    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-data\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708150 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.708117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-certs\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.708340 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.708320    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e7f7f-834a-402c-9d96-295ac56f2d20-istiod-ca-cert\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.710231 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.710210    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-envoy\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.710422 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.710400    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-podinfo\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.715142 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.715122    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-token\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.715255 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.715147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d7jr\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-kube-api-access-5d7jr\") pod \"e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") " pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.813375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.813301    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:15.840178 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:15.840143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:16.090623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:16.090512    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"]
Sep 06 01:16:16.093340 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:16:16.093305    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347e7f7f_834a_402c_9d96_295ac56f2d20.slice/crio-2aa16bb2cc057d0a309b01dd24e6466865c001478ff0ff36b031cfbfabcc5f78 WatchSource:0}: Error finding container 2aa16bb2cc057d0a309b01dd24e6466865c001478ff0ff36b031cfbfabcc5f78: Status 404 returned error can't find the container with id 2aa16bb2cc057d0a309b01dd24e6466865c001478ff0ff36b031cfbfabcc5f78
Sep 06 01:16:16.095528 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:16.095497    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:16:16.095598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:16.095562    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:16:16.095598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:16.095589    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:16:16.983924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:16.983888    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" event={"ID":"347e7f7f-834a-402c-9d96-295ac56f2d20","Type":"ContainerStarted","Data":"6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"}
Sep 06 01:16:16.984389 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:16.983965    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" event={"ID":"347e7f7f-834a-402c-9d96-295ac56f2d20","Type":"ContainerStarted","Data":"2aa16bb2cc057d0a309b01dd24e6466865c001478ff0ff36b031cfbfabcc5f78"}
Sep 06 01:16:17.005038 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:17.004991    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podStartSLOduration=2.004978631 podStartE2EDuration="2.004978631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:16:17.001879077 +0000 UTC m=+1539.331237915" watchObservedRunningTime="2026-09-06 01:16:17.004978631 +0000 UTC m=+1539.334337466"
Sep 06 01:16:17.749758 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:17.749691    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:17.813888 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:17.813841    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:17.819009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:17.818985    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:17.990137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:17.990112    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:17.991476 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:17.991457    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:18.314132 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.314100    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:18.354560 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.354530    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"]
Sep 06 01:16:18.354799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.354773    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs" podUID="26375bb7-5b82-4863-b63c-cddfd30e1a59" containerName="payload-processing" containerID="cri-o://3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f" gracePeriod=30
Sep 06 01:16:18.364159 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.364134    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"]
Sep 06 01:16:18.364334 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.364315    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg" podUID="9c93615d-fca7-4a33-b242-e2bf0319a39b" containerName="payload-pre-processing" containerID="cri-o://682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc" gracePeriod=30
Sep 06 01:16:18.401623 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.401576    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:18.463059 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.463035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:18.517836 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.517799    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"]
Sep 06 01:16:18.520366 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.520338    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e6b4e2-kcqqx"]
Sep 06 01:16:18.525091 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.525063    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:18.629478 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.629453    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:16:18.646065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.646042    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:16:18.742178 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742138    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c93615d-fca7-4a33-b242-e2bf0319a39b-plugins-config-volume\") pod \"9c93615d-fca7-4a33-b242-e2bf0319a39b\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") "
Sep 06 01:16:18.742355 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742202    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgtn6\" (UniqueName: \"kubernetes.io/projected/26375bb7-5b82-4863-b63c-cddfd30e1a59-kube-api-access-bgtn6\") pod \"26375bb7-5b82-4863-b63c-cddfd30e1a59\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") "
Sep 06 01:16:18.742355 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742249    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9mxt\" (UniqueName: \"kubernetes.io/projected/9c93615d-fca7-4a33-b242-e2bf0319a39b-kube-api-access-b9mxt\") pod \"9c93615d-fca7-4a33-b242-e2bf0319a39b\" (UID: \"9c93615d-fca7-4a33-b242-e2bf0319a39b\") "
Sep 06 01:16:18.742355 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742298    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26375bb7-5b82-4863-b63c-cddfd30e1a59-plugins-config-volume\") pod \"26375bb7-5b82-4863-b63c-cddfd30e1a59\" (UID: \"26375bb7-5b82-4863-b63c-cddfd30e1a59\") "
Sep 06 01:16:18.742525 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742494    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c93615d-fca7-4a33-b242-e2bf0319a39b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c93615d-fca7-4a33-b242-e2bf0319a39b" (UID: "9c93615d-fca7-4a33-b242-e2bf0319a39b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:18.742741 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742719    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c93615d-fca7-4a33-b242-e2bf0319a39b-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:18.742741 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.742726    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26375bb7-5b82-4863-b63c-cddfd30e1a59-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26375bb7-5b82-4863-b63c-cddfd30e1a59" (UID: "26375bb7-5b82-4863-b63c-cddfd30e1a59"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:18.744464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.744446    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26375bb7-5b82-4863-b63c-cddfd30e1a59-kube-api-access-bgtn6" (OuterVolumeSpecName: "kube-api-access-bgtn6") pod "26375bb7-5b82-4863-b63c-cddfd30e1a59" (UID: "26375bb7-5b82-4863-b63c-cddfd30e1a59"). InnerVolumeSpecName "kube-api-access-bgtn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:18.744520 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.744461    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c93615d-fca7-4a33-b242-e2bf0319a39b-kube-api-access-b9mxt" (OuterVolumeSpecName: "kube-api-access-b9mxt") pod "9c93615d-fca7-4a33-b242-e2bf0319a39b" (UID: "9c93615d-fca7-4a33-b242-e2bf0319a39b"). InnerVolumeSpecName "kube-api-access-b9mxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:18.843360 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.843281    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgtn6\" (UniqueName: \"kubernetes.io/projected/26375bb7-5b82-4863-b63c-cddfd30e1a59-kube-api-access-bgtn6\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:18.843360 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.843311    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9mxt\" (UniqueName: \"kubernetes.io/projected/9c93615d-fca7-4a33-b242-e2bf0319a39b-kube-api-access-b9mxt\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:18.843360 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.843326    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26375bb7-5b82-4863-b63c-cddfd30e1a59-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:18.874844 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.874818    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:18.983760 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.983731    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:18.997244 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.997214    2575 generic.go:356] "Generic (PLEG): container finished" podID="9c93615d-fca7-4a33-b242-e2bf0319a39b" containerID="682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc" exitCode=0
Sep 06 01:16:18.997626 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.997258    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"
Sep 06 01:16:18.997626 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.997311    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg" event={"ID":"9c93615d-fca7-4a33-b242-e2bf0319a39b","Type":"ContainerDied","Data":"682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc"}
Sep 06 01:16:18.997626 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.997364    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg" event={"ID":"9c93615d-fca7-4a33-b242-e2bf0319a39b","Type":"ContainerDied","Data":"4e2d1cf6729948a597b657b1f4aea6430fec39804dbf94cf1f99b8604818f61a"}
Sep 06 01:16:18.997626 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.997397    2575 scope.go:117] "RemoveContainer" containerID="682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc"
Sep 06 01:16:18.998706 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.998685    2575 generic.go:356] "Generic (PLEG): container finished" podID="26375bb7-5b82-4863-b63c-cddfd30e1a59" containerID="3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f" exitCode=0
Sep 06 01:16:18.998761 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.998735    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"
Sep 06 01:16:18.998839 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.998819    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs" event={"ID":"26375bb7-5b82-4863-b63c-cddfd30e1a59","Type":"ContainerDied","Data":"3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f"}
Sep 06 01:16:18.998913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:18.998879    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs" event={"ID":"26375bb7-5b82-4863-b63c-cddfd30e1a59","Type":"ContainerDied","Data":"1bf3d02f0e29ba4d33f66507260e38d7e3095cf29759a2145dae047ca5f7ddb9"}
Sep 06 01:16:19.009065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.009045    2575 scope.go:117] "RemoveContainer" containerID="682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc"
Sep 06 01:16:19.009377 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:19.009349    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc\": container with ID starting with 682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc not found: ID does not exist" containerID="682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc"
Sep 06 01:16:19.009493 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.009387    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc"} err="failed to get container status \"682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc\": rpc error: code = NotFound desc = could not find container \"682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc\": container with ID starting with 682cda09fe199adc979e766c8270776b6428ff87cd0f2f0647b0a840288dedcc not found: ID does not exist"
Sep 06 01:16:19.009493 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.009411    2575 scope.go:117] "RemoveContainer" containerID="3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f"
Sep 06 01:16:19.020354 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.020330    2575 scope.go:117] "RemoveContainer" containerID="3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f"
Sep 06 01:16:19.020825 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:19.020802    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f\": container with ID starting with 3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f not found: ID does not exist" containerID="3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f"
Sep 06 01:16:19.020961 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.020834    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f"} err="failed to get container status \"3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f\": rpc error: code = NotFound desc = could not find container \"3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f\": container with ID starting with 3752a25f11a876357ab5335d97ca01c3592b4d879930ee50b0024415b512ab9f not found: ID does not exist"
Sep 06 01:16:19.031605 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.031585    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"]
Sep 06 01:16:19.037453 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.037432    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e6b4e2-fdcb8bb8b-7wfwg"]
Sep 06 01:16:19.047237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.047213    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"]
Sep 06 01:16:19.053463 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.053446    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e6b4e2-df8c7b989-2vpcs"]
Sep 06 01:16:19.056375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:19.056355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:20.203205 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.203167    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26375bb7-5b82-4863-b63c-cddfd30e1a59" path="/var/lib/kubelet/pods/26375bb7-5b82-4863-b63c-cddfd30e1a59/volumes"
Sep 06 01:16:20.203672 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.203655    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c93615d-fca7-4a33-b242-e2bf0319a39b" path="/var/lib/kubelet/pods/9c93615d-fca7-4a33-b242-e2bf0319a39b/volumes"
Sep 06 01:16:20.203999 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.203985    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e566481c-3bee-447e-8715-dbebcdfc7b63" path="/var/lib/kubelet/pods/e566481c-3bee-447e-8715-dbebcdfc7b63/volumes"
Sep 06 01:16:20.242080 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.242052    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:20.337395 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.337365    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:20.520620 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.520522    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:20.730664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.730631    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:20.815591 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.815493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:20.817255 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817235    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"]
Sep 06 01:16:20.817627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817613    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26375bb7-5b82-4863-b63c-cddfd30e1a59" containerName="payload-processing"
Sep 06 01:16:20.817676 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817630    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="26375bb7-5b82-4863-b63c-cddfd30e1a59" containerName="payload-processing"
Sep 06 01:16:20.817676 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817658    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c93615d-fca7-4a33-b242-e2bf0319a39b" containerName="payload-pre-processing"
Sep 06 01:16:20.817676 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817663    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c93615d-fca7-4a33-b242-e2bf0319a39b" containerName="payload-pre-processing"
Sep 06 01:16:20.817784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817713    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="26375bb7-5b82-4863-b63c-cddfd30e1a59" containerName="payload-processing"
Sep 06 01:16:20.817784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.817723    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c93615d-fca7-4a33-b242-e2bf0319a39b" containerName="payload-pre-processing"
Sep 06 01:16:20.822155 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.822139    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:20.829725 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.829697    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-074d50\""
Sep 06 01:16:20.834184 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.834161    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"]
Sep 06 01:16:20.857757 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.857731    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"]
Sep 06 01:16:20.859399 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.859373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76wkb\" (UniqueName: \"kubernetes.io/projected/50939f74-fa59-45d0-84f0-4327f4623b0a-kube-api-access-76wkb\") pod \"payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:20.859485 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.859426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50939f74-fa59-45d0-84f0-4327f4623b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:20.861304 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.861285    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:20.863674 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.863654    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-074d50-dockercfg-sthtf\""
Sep 06 01:16:20.873814 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.873792    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"]
Sep 06 01:16:20.961118 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.961080    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50939f74-fa59-45d0-84f0-4327f4623b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:20.961320 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.961244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec456-04a8-4a07-bc8d-1623930eed24-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:20.961415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.961327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76wkb\" (UniqueName: \"kubernetes.io/projected/50939f74-fa59-45d0-84f0-4327f4623b0a-kube-api-access-76wkb\") pod \"payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:20.961415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.961353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgzpf\" (UniqueName: \"kubernetes.io/projected/1e4ec456-04a8-4a07-bc8d-1623930eed24-kube-api-access-zgzpf\") pod \"payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:20.961984 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.961950    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50939f74-fa59-45d0-84f0-4327f4623b0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:20.970138 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:20.970111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76wkb\" (UniqueName: \"kubernetes.io/projected/50939f74-fa59-45d0-84f0-4327f4623b0a-kube-api-access-76wkb\") pod \"payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:21.062194 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.062167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec456-04a8-4a07-bc8d-1623930eed24-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:21.062294 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.062226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgzpf\" (UniqueName: \"kubernetes.io/projected/1e4ec456-04a8-4a07-bc8d-1623930eed24-kube-api-access-zgzpf\") pod \"payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:21.062813 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.062796    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec456-04a8-4a07-bc8d-1623930eed24-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:21.070003 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.069933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgzpf\" (UniqueName: \"kubernetes.io/projected/1e4ec456-04a8-4a07-bc8d-1623930eed24-kube-api-access-zgzpf\") pod \"payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:21.139730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.139698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:21.171495 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.171469    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:21.412669 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.412645    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"]
Sep 06 01:16:21.413661 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:16:21.413635    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50939f74_fa59_45d0_84f0_4327f4623b0a.slice/crio-b799cffd2055b2c5f1fd1e717da1aa737b4e19347848634ca0cfc1d607a6f999 WatchSource:0}: Error finding container b799cffd2055b2c5f1fd1e717da1aa737b4e19347848634ca0cfc1d607a6f999: Status 404 returned error can't find the container with id b799cffd2055b2c5f1fd1e717da1aa737b4e19347848634ca0cfc1d607a6f999
Sep 06 01:16:21.453512 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:21.453486    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"]
Sep 06 01:16:21.454471 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:16:21.454448    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4ec456_04a8_4a07_bc8d_1623930eed24.slice/crio-cabe09df3812b68cd0c2b947248cadac20a3dd63509a1ee26f0d43dc0747ff9e WatchSource:0}: Error finding container cabe09df3812b68cd0c2b947248cadac20a3dd63509a1ee26f0d43dc0747ff9e: Status 404 returned error can't find the container with id cabe09df3812b68cd0c2b947248cadac20a3dd63509a1ee26f0d43dc0747ff9e
Sep 06 01:16:22.020211 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.020177    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg" event={"ID":"1e4ec456-04a8-4a07-bc8d-1623930eed24","Type":"ContainerStarted","Data":"1320e832015493c9a1453597b0f3e173c71f5842b33428cf10eb5a73c2112b66"}
Sep 06 01:16:22.020418 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.020234    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg" event={"ID":"1e4ec456-04a8-4a07-bc8d-1623930eed24","Type":"ContainerStarted","Data":"cabe09df3812b68cd0c2b947248cadac20a3dd63509a1ee26f0d43dc0747ff9e"}
Sep 06 01:16:22.020418 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.020306    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:22.021913 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.021891    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4" event={"ID":"50939f74-fa59-45d0-84f0-4327f4623b0a","Type":"ContainerStarted","Data":"55a2685de9aa71a6437629a689edd3099c148c5743500144613e7881e221f227"}
Sep 06 01:16:22.022017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.021924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4" event={"ID":"50939f74-fa59-45d0-84f0-4327f4623b0a","Type":"ContainerStarted","Data":"b799cffd2055b2c5f1fd1e717da1aa737b4e19347848634ca0cfc1d607a6f999"}
Sep 06 01:16:22.022017 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.021977    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:22.035264 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.035209    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg" podStartSLOduration=2.035192818 podStartE2EDuration="2.035192818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:16:22.034347082 +0000 UTC m=+1544.363705928" watchObservedRunningTime="2026-09-06 01:16:22.035192818 +0000 UTC m=+1544.364551655"
Sep 06 01:16:22.051269 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:22.051234    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4" podStartSLOduration=2.051222629 podStartE2EDuration="2.051222629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:16:22.048746925 +0000 UTC m=+1544.378105789" watchObservedRunningTime="2026-09-06 01:16:22.051222629 +0000 UTC m=+1544.380581464"
Sep 06 01:16:24.996673 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:24.996641    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:25.098539 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.098504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:25.264503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.264414    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"]
Sep 06 01:16:25.264656 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.264599    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy" containerID="cri-o://787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d" gracePeriod=30
Sep 06 01:16:25.288333 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.288303    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:25.654240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.654163    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"]
Sep 06 01:16:25.660882 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.660531    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:25.663729 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.663703    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:16:25.664396 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.663816    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:16:25.667720 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.667701    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"]
Sep 06 01:16:25.706436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.706409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368949e9-31e8-4fff-9b42-61ec5ecbdf54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn\" (UID: \"368949e9-31e8-4fff-9b42-61ec5ecbdf54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:25.767170 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.767121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:25.806964 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.806936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368949e9-31e8-4fff-9b42-61ec5ecbdf54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn\" (UID: \"368949e9-31e8-4fff-9b42-61ec5ecbdf54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:25.807533 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.807510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368949e9-31e8-4fff-9b42-61ec5ecbdf54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn\" (UID: \"368949e9-31e8-4fff-9b42-61ec5ecbdf54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:25.973184 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:25.973151    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:26.250678 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:26.250655    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"]
Sep 06 01:16:26.252324 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:16:26.252293    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368949e9_31e8_4fff_9b42_61ec5ecbdf54.slice/crio-efa7dfa02ed20b03a4bde9b779e841f9d642dd785ea90fb4d4a9d397670e108c WatchSource:0}: Error finding container efa7dfa02ed20b03a4bde9b779e841f9d642dd785ea90fb4d4a9d397670e108c: Status 404 returned error can't find the container with id efa7dfa02ed20b03a4bde9b779e841f9d642dd785ea90fb4d4a9d397670e108c
Sep 06 01:16:27.055039 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:27.055005    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn" event={"ID":"368949e9-31e8-4fff-9b42-61ec5ecbdf54","Type":"ContainerStarted","Data":"4839dd6852c0988864b1ea4fff339f9b20f1b9e9f310f08577994e4b72f78a9f"}
Sep 06 01:16:27.055169 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:27.055070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn" event={"ID":"368949e9-31e8-4fff-9b42-61ec5ecbdf54","Type":"ContainerStarted","Data":"efa7dfa02ed20b03a4bde9b779e841f9d642dd785ea90fb4d4a9d397670e108c"}
Sep 06 01:16:27.076709 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:27.076664    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn" podStartSLOduration=2.076650475 podStartE2EDuration="2.076650475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:16:27.073274809 +0000 UTC m=+1549.402633645" watchObservedRunningTime="2026-09-06 01:16:27.076650475 +0000 UTC m=+1549.406009311"
Sep 06 01:16:28.063647 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:28.063616    2575 generic.go:356] "Generic (PLEG): container finished" podID="368949e9-31e8-4fff-9b42-61ec5ecbdf54" containerID="4839dd6852c0988864b1ea4fff339f9b20f1b9e9f310f08577994e4b72f78a9f" exitCode=0
Sep 06 01:16:28.064006 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:28.063701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn" event={"ID":"368949e9-31e8-4fff-9b42-61ec5ecbdf54","Type":"ContainerDied","Data":"4839dd6852c0988864b1ea4fff339f9b20f1b9e9f310f08577994e4b72f78a9f"}
Sep 06 01:16:29.207041 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:29.207010    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:29.340642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:29.340543    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368949e9-31e8-4fff-9b42-61ec5ecbdf54-maas-api-service-ca\") pod \"368949e9-31e8-4fff-9b42-61ec5ecbdf54\" (UID: \"368949e9-31e8-4fff-9b42-61ec5ecbdf54\") "
Sep 06 01:16:29.340929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:29.340901    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368949e9-31e8-4fff-9b42-61ec5ecbdf54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "368949e9-31e8-4fff-9b42-61ec5ecbdf54" (UID: "368949e9-31e8-4fff-9b42-61ec5ecbdf54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:29.441926 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:29.441888    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/368949e9-31e8-4fff-9b42-61ec5ecbdf54-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.077871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.077828    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"
Sep 06 01:16:30.077871 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.077837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn" event={"ID":"368949e9-31e8-4fff-9b42-61ec5ecbdf54","Type":"ContainerDied","Data":"efa7dfa02ed20b03a4bde9b779e841f9d642dd785ea90fb4d4a9d397670e108c"}
Sep 06 01:16:30.078252 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.077897    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efa7dfa02ed20b03a4bde9b779e841f9d642dd785ea90fb4d4a9d397670e108c"
Sep 06 01:16:30.540870 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.540827    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:16:30.658804 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.658706    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e50a7323-6cad-4ae0-887b-67c813061e9f-istiod-ca-cert\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659084 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659060    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-podinfo\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659206 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659192    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-certs\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659293    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bddq7\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-kube-api-access-bddq7\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659480 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659467    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-envoy\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659605 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659592    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-token\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659706    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-data\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659827 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659816    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-socket\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.659975 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.659960    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-credential-socket\") pod \"e50a7323-6cad-4ae0-887b-67c813061e9f\" (UID: \"e50a7323-6cad-4ae0-887b-67c813061e9f\") "
Sep 06 01:16:30.660620 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.660600    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:30.661067 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.661048    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50a7323-6cad-4ae0-887b-67c813061e9f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:30.665294 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.665014    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:16:30.668235 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.668082    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:30.668527 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.668377    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:30.668927 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.668649    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:30.678923 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.678884    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:16:30.680122 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.679966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-kube-api-access-bddq7" (OuterVolumeSpecName: "kube-api-access-bddq7") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "kube-api-access-bddq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:30.680122 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.679985    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e50a7323-6cad-4ae0-887b-67c813061e9f" (UID: "e50a7323-6cad-4ae0-887b-67c813061e9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:30.761664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761618    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761664    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761676    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761689    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761700    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e50a7323-6cad-4ae0-887b-67c813061e9f-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761739    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761750    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761761    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bddq7\" (UniqueName: \"kubernetes.io/projected/e50a7323-6cad-4ae0-887b-67c813061e9f-kube-api-access-bddq7\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:30.761951 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:30.761772    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e50a7323-6cad-4ae0-887b-67c813061e9f-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:31.085648 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.085607    2575 generic.go:356] "Generic (PLEG): container finished" podID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerID="787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d" exitCode=0
Sep 06 01:16:31.085648 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.085650    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"
Sep 06 01:16:31.085890 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.085697    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" event={"ID":"e50a7323-6cad-4ae0-887b-67c813061e9f","Type":"ContainerDied","Data":"787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d"}
Sep 06 01:16:31.085890 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.085782    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656" event={"ID":"e50a7323-6cad-4ae0-887b-67c813061e9f","Type":"ContainerDied","Data":"02b84a4033af7859dd560f44e5ed2b4ae77e425c5c6d1c0ef0e320592476dc46"}
Sep 06 01:16:31.085890 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.085809    2575 scope.go:117] "RemoveContainer" containerID="787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d"
Sep 06 01:16:31.095750 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.095729    2575 scope.go:117] "RemoveContainer" containerID="597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"
Sep 06 01:16:31.104394 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.104376    2575 scope.go:117] "RemoveContainer" containerID="787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d"
Sep 06 01:16:31.104679 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:31.104660    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d\": container with ID starting with 787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d not found: ID does not exist" containerID="787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d"
Sep 06 01:16:31.104727 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.104688    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d"} err="failed to get container status \"787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d\": rpc error: code = NotFound desc = could not find container \"787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d\": container with ID starting with 787e78039937faeb933802c7c5802344884a934aad9d516d24f3821b72b59b8d not found: ID does not exist"
Sep 06 01:16:31.104727 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.104706    2575 scope.go:117] "RemoveContainer" containerID="597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"
Sep 06 01:16:31.104992 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:31.104978    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0\": container with ID starting with 597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0 not found: ID does not exist" containerID="597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"
Sep 06 01:16:31.105035 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.104997    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0"} err="failed to get container status \"597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0\": rpc error: code = NotFound desc = could not find container \"597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0\": container with ID starting with 597256467fcad47a1479c81fbeba1bc21dcf88ebd88ab48a6a798cae28088fc0 not found: ID does not exist"
Sep 06 01:16:31.110696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.110661    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"]
Sep 06 01:16:31.113065 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:31.113046    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e6b4e2-openshift-default-66876c8cc4-5v656"]
Sep 06 01:16:32.202045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:32.202010    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" path="/var/lib/kubelet/pods/e50a7323-6cad-4ae0-887b-67c813061e9f/volumes"
Sep 06 01:16:33.030922 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:33.030893    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:16:33.031108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:33.030979    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:16:36.242459 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.242412    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:36.275111 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.275052    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"]
Sep 06 01:16:36.275346 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.275316    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv" podUID="e35434d4-78d5-428b-8075-99299c8bfda7" containerName="payload-pre-processing" containerID="cri-o://3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f" gracePeriod=30
Sep 06 01:16:36.345439 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.345408    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:36.446149 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.446113    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:36.468924 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.468881    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"]
Sep 06 01:16:36.472988 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.472958    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-812148-sd7dn"]
Sep 06 01:16:36.536838 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.536813    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:16:36.543490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.543464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:36.621929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.621895    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbwpm\" (UniqueName: \"kubernetes.io/projected/e35434d4-78d5-428b-8075-99299c8bfda7-kube-api-access-tbwpm\") pod \"e35434d4-78d5-428b-8075-99299c8bfda7\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") "
Sep 06 01:16:36.622121 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.621977    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e35434d4-78d5-428b-8075-99299c8bfda7-plugins-config-volume\") pod \"e35434d4-78d5-428b-8075-99299c8bfda7\" (UID: \"e35434d4-78d5-428b-8075-99299c8bfda7\") "
Sep 06 01:16:36.622415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.622394    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e35434d4-78d5-428b-8075-99299c8bfda7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e35434d4-78d5-428b-8075-99299c8bfda7" (UID: "e35434d4-78d5-428b-8075-99299c8bfda7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:16:36.624363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.624341    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e35434d4-78d5-428b-8075-99299c8bfda7-kube-api-access-tbwpm" (OuterVolumeSpecName: "kube-api-access-tbwpm") pod "e35434d4-78d5-428b-8075-99299c8bfda7" (UID: "e35434d4-78d5-428b-8075-99299c8bfda7"). InnerVolumeSpecName "kube-api-access-tbwpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:16:36.723482 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.723438    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbwpm\" (UniqueName: \"kubernetes.io/projected/e35434d4-78d5-428b-8075-99299c8bfda7-kube-api-access-tbwpm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:36.723482 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.723478    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e35434d4-78d5-428b-8075-99299c8bfda7-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:16:36.978993 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:36.978962    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:37.126919 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.126879    2575 generic.go:356] "Generic (PLEG): container finished" podID="e35434d4-78d5-428b-8075-99299c8bfda7" containerID="3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f" exitCode=0
Sep 06 01:16:37.127103 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.126991    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"
Sep 06 01:16:37.127103 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.127015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv" event={"ID":"e35434d4-78d5-428b-8075-99299c8bfda7","Type":"ContainerDied","Data":"3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f"}
Sep 06 01:16:37.127103 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.127078    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv" event={"ID":"e35434d4-78d5-428b-8075-99299c8bfda7","Type":"ContainerDied","Data":"c275b8a2ee619c416cd6329be9c88670154a4b1d5f6669ccefb6028af003dac0"}
Sep 06 01:16:37.127267 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.127109    2575 scope.go:117] "RemoveContainer" containerID="3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f"
Sep 06 01:16:37.139374 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.139351    2575 scope.go:117] "RemoveContainer" containerID="3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f"
Sep 06 01:16:37.139674 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:16:37.139650    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f\": container with ID starting with 3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f not found: ID does not exist" containerID="3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f"
Sep 06 01:16:37.139756 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.139684    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f"} err="failed to get container status \"3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f\": rpc error: code = NotFound desc = could not find container \"3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f\": container with ID starting with 3b657f83df841ecc2db00cd3b73481ee382861722a1d8ff068230cc604e3339f not found: ID does not exist"
Sep 06 01:16:37.149323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.149297    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"]
Sep 06 01:16:37.153385 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:37.153362    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-812148-665d85598b-fc7fv"]
Sep 06 01:16:38.204094 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:38.204060    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="368949e9-31e8-4fff-9b42-61ec5ecbdf54" path="/var/lib/kubelet/pods/368949e9-31e8-4fff-9b42-61ec5ecbdf54/volumes"
Sep 06 01:16:38.204628 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:38.204605    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e35434d4-78d5-428b-8075-99299c8bfda7" path="/var/lib/kubelet/pods/e35434d4-78d5-428b-8075-99299c8bfda7/volumes"
Sep 06 01:16:39.140985 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:39.140897    2575 generic.go:356] "Generic (PLEG): container finished" podID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerID="6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b" exitCode=0
Sep 06 01:16:39.140985 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:39.140972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" event={"ID":"347e7f7f-834a-402c-9d96-295ac56f2d20","Type":"ContainerDied","Data":"6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"}
Sep 06 01:16:39.141430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:39.141412    2575 scope.go:117] "RemoveContainer" containerID="6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"
Sep 06 01:16:39.142563 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:39.142530    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:16:39.142671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:39.142598    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:16:39.142671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:39.142631    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Sep 06 01:16:40.168816 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:40.168763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" event={"ID":"347e7f7f-834a-402c-9d96-295ac56f2d20","Type":"ContainerStarted","Data":"0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20"}
Sep 06 01:16:40.813776 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:40.813734    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:40.814040 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:40.814015    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Sep 06 01:16:40.814130 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:40.814075    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Sep 06 01:16:41.126234 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:41.126144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:41.814347 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:41.814308    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Sep 06 01:16:41.814738 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:41.814384    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Sep 06 01:16:42.814397 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:42.814362    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Sep 06 01:16:42.814808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:42.814415    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Sep 06 01:16:43.377986 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:43.377943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:43.477426 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:43.477397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:43.657166 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:43.657068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:43.814235 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:43.814198    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Sep 06 01:16:43.814441 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:43.814261    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Sep 06 01:16:44.005139 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:44.005104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:44.814167 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:44.814132    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Sep 06 01:16:44.814349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:44.814186    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Sep 06 01:16:45.818330 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:45.818297    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:45.818695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:45.818565    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:45.819299 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:45.819278    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:16:49.224783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:49.224750    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-rjs67_95231315-9b02-4114-9976-6af33cfc6af9/payload-processing/0.log"
Sep 06 01:16:49.351891 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:49.351863    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg_1e4ec456-04a8-4a07-bc8d-1623930eed24/payload-processing/0.log"
Sep 06 01:16:56.063137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063094    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"]
Sep 06 01:16:56.063523 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063504    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e35434d4-78d5-428b-8075-99299c8bfda7" containerName="payload-pre-processing"
Sep 06 01:16:56.063523 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063516    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e35434d4-78d5-428b-8075-99299c8bfda7" containerName="payload-pre-processing"
Sep 06 01:16:56.063602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063527    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368949e9-31e8-4fff-9b42-61ec5ecbdf54" containerName="revoke-keys"
Sep 06 01:16:56.063602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063533    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="368949e9-31e8-4fff-9b42-61ec5ecbdf54" containerName="revoke-keys"
Sep 06 01:16:56.063602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063541    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy"
Sep 06 01:16:56.063602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063546    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy"
Sep 06 01:16:56.063602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063553    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy"
Sep 06 01:16:56.063602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063558    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy"
Sep 06 01:16:56.063791 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063616    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="368949e9-31e8-4fff-9b42-61ec5ecbdf54" containerName="revoke-keys"
Sep 06 01:16:56.063791 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063624    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy"
Sep 06 01:16:56.063791 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063630    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e35434d4-78d5-428b-8075-99299c8bfda7" containerName="payload-pre-processing"
Sep 06 01:16:56.063791 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.063635    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50a7323-6cad-4ae0-887b-67c813061e9f" containerName="istio-proxy"
Sep 06 01:16:56.068055 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.068039    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.070755 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.070732    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b2e4f6\"/\"openshift-service-ca.crt\""
Sep 06 01:16:56.070912 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.070770    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b2e4f6\"/\"kube-root-ca.crt\""
Sep 06 01:16:56.071501 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.071487    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b2e4f6\"/\"default-dockercfg-pthw2\""
Sep 06 01:16:56.071577 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.071559    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b2e4f6\"/\"ipp-route-b2e4f6-kserve-self-signed-certs\""
Sep 06 01:16:56.077123 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.077102    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"]
Sep 06 01:16:56.112183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.112159    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-model-cache\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.112293 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.112204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-home\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.112293 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.112225    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-dshm\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.112293 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.112272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tmp-dir\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.112400 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.112293    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptbw\" (UniqueName: \"kubernetes.io/projected/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-kube-api-access-wptbw\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.112400 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.112321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tls-certs\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.167347 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.167316    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:16:56.213697 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.213670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tls-certs\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.213816 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.213724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-model-cache\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.213816 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.213764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-home\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.213816 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.213784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-dshm\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.214024 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.213816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tmp-dir\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.214024 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.213846    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptbw\" (UniqueName: \"kubernetes.io/projected/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-kube-api-access-wptbw\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.214241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.214216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tmp-dir\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.214339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.214243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-model-cache\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.214339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.214284    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-home\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.216172 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.216153    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-dshm\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.216349 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.216333    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tls-certs\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.221751 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.221732    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptbw\" (UniqueName: \"kubernetes.io/projected/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-kube-api-access-wptbw\") pod \"ipp-route-b2e4f6-kserve-77987654f5-pcz8m\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") " pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.378461 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.378371    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:16:56.654396 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:56.654372    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"]
Sep 06 01:16:56.656210 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:16:56.656187    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d405c8b_2f60_4d0c_b669_f3d6c8f0cc8c.slice/crio-6d0958084c0137af3adfdbfc4bcd743a384fd1d3781233221e81bf08032b5c42 WatchSource:0}: Error finding container 6d0958084c0137af3adfdbfc4bcd743a384fd1d3781233221e81bf08032b5c42: Status 404 returned error can't find the container with id 6d0958084c0137af3adfdbfc4bcd743a384fd1d3781233221e81bf08032b5c42
Sep 06 01:16:57.273261 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:57.273227    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m" event={"ID":"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c","Type":"ContainerStarted","Data":"d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4"}
Sep 06 01:16:57.273650 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:57.273270    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m" event={"ID":"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c","Type":"ContainerStarted","Data":"6d0958084c0137af3adfdbfc4bcd743a384fd1d3781233221e81bf08032b5c42"}
Sep 06 01:16:57.290624 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:16:57.290572    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m" podStartSLOduration=1.290557733 podStartE2EDuration="1.290557733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:16:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:16:57.289506198 +0000 UTC m=+1579.618865034" watchObservedRunningTime="2026-09-06 01:16:57.290557733 +0000 UTC m=+1579.619916849"
Sep 06 01:17:06.378940 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:06.378910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:17:06.379476 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:06.379058    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:17:06.391722 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:06.391700    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:17:07.342318 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:07.342287    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:17:24.203450 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:24.203416    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c_e40dfedf-864c-41cb-8a72-5ad7de2908fb/payload-processing/0.log"
Sep 06 01:17:24.342461 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:24.342424    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-rjs67_95231315-9b02-4114-9976-6af33cfc6af9/payload-processing/0.log"
Sep 06 01:17:24.475221 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:24.475142    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg_1e4ec456-04a8-4a07-bc8d-1623930eed24/payload-processing/0.log"
Sep 06 01:17:24.777618 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:24.777541    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:25.099695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:25.099620    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:25.221505 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:25.221473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:28.132114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:28.132078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:28.228534 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:28.228502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:28.671579 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:28.671545    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:29.445103 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:29.445066    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:29.542891 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:29.542843    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:29.618031 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:29.617996    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:29.966926 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:29.966892    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:30.109581 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.109544    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:30.167173 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.167141    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"]
Sep 06 01:17:30.170909 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.170891    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.173382 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.173360    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-3032be-dockercfg-mm9qt\""
Sep 06 01:17:30.173497 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.173361    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-3032be\""
Sep 06 01:17:30.181762 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.181740    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"]
Sep 06 01:17:30.255802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.255718    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"]
Sep 06 01:17:30.259820 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.259752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.271439 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.271412    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"]
Sep 06 01:17:30.328296 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.328267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8987733-c3cb-4280-8549-103ab312a952-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.328467 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.328350    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msxsf\" (UniqueName: \"kubernetes.io/projected/c8987733-c3cb-4280-8549-103ab312a952-kube-api-access-msxsf\") pod \"payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.429205 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.429171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msxsf\" (UniqueName: \"kubernetes.io/projected/c8987733-c3cb-4280-8549-103ab312a952-kube-api-access-msxsf\") pod \"payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.429363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.429221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm5dx\" (UniqueName: \"kubernetes.io/projected/8adcd102-0c8f-49ce-85cd-8f7921793fdd-kube-api-access-qm5dx\") pod \"payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.429363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.429259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8adcd102-0c8f-49ce-85cd-8f7921793fdd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.429363 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.429323    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8987733-c3cb-4280-8549-103ab312a952-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.430137 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.430105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8987733-c3cb-4280-8549-103ab312a952-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.437325 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.437300    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msxsf\" (UniqueName: \"kubernetes.io/projected/c8987733-c3cb-4280-8549-103ab312a952-kube-api-access-msxsf\") pod \"payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.482288 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.482265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:30.531292 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.530758    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm5dx\" (UniqueName: \"kubernetes.io/projected/8adcd102-0c8f-49ce-85cd-8f7921793fdd-kube-api-access-qm5dx\") pod \"payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.531292 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.531237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8adcd102-0c8f-49ce-85cd-8f7921793fdd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.532227 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.532200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8adcd102-0c8f-49ce-85cd-8f7921793fdd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.540746 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.540714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm5dx\" (UniqueName: \"kubernetes.io/projected/8adcd102-0c8f-49ce-85cd-8f7921793fdd-kube-api-access-qm5dx\") pod \"payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.571507 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.571482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:30.755159 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.755129    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"]
Sep 06 01:17:30.756787 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:17:30.756764    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8987733_c3cb_4280_8549_103ab312a952.slice/crio-d27db83b2adcbe19445e707a53e7fb04fc3b41a0f72a459b9cb4e33ec3256f5d WatchSource:0}: Error finding container d27db83b2adcbe19445e707a53e7fb04fc3b41a0f72a459b9cb4e33ec3256f5d: Status 404 returned error can't find the container with id d27db83b2adcbe19445e707a53e7fb04fc3b41a0f72a459b9cb4e33ec3256f5d
Sep 06 01:17:30.845765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:30.845739    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"]
Sep 06 01:17:30.846710 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:17:30.846685    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8adcd102_0c8f_49ce_85cd_8f7921793fdd.slice/crio-a2f8bfc9e184492f9ec7d3dbdfc607ca073382a6eca087ac9ddb29d7a357c108 WatchSource:0}: Error finding container a2f8bfc9e184492f9ec7d3dbdfc607ca073382a6eca087ac9ddb29d7a357c108: Status 404 returned error can't find the container with id a2f8bfc9e184492f9ec7d3dbdfc607ca073382a6eca087ac9ddb29d7a357c108
Sep 06 01:17:31.478284 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.478245    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt" event={"ID":"8adcd102-0c8f-49ce-85cd-8f7921793fdd","Type":"ContainerStarted","Data":"b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446"}
Sep 06 01:17:31.478499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.478310    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt" event={"ID":"8adcd102-0c8f-49ce-85cd-8f7921793fdd","Type":"ContainerStarted","Data":"a2f8bfc9e184492f9ec7d3dbdfc607ca073382a6eca087ac9ddb29d7a357c108"}
Sep 06 01:17:31.478499 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.478442    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:31.480023 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.479995    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l" event={"ID":"c8987733-c3cb-4280-8549-103ab312a952","Type":"ContainerStarted","Data":"913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404"}
Sep 06 01:17:31.480151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.480042    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l" event={"ID":"c8987733-c3cb-4280-8549-103ab312a952","Type":"ContainerStarted","Data":"d27db83b2adcbe19445e707a53e7fb04fc3b41a0f72a459b9cb4e33ec3256f5d"}
Sep 06 01:17:31.480151 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.480070    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:31.499772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.499723    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt" podStartSLOduration=1.499708294 podStartE2EDuration="1.499708294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:17:31.497529147 +0000 UTC m=+1613.826888009" watchObservedRunningTime="2026-09-06 01:17:31.499708294 +0000 UTC m=+1613.829067140"
Sep 06 01:17:31.514464 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:31.514418    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l" podStartSLOduration=1.514402417 podStartE2EDuration="1.514402417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:17:31.513350943 +0000 UTC m=+1613.842709800" watchObservedRunningTime="2026-09-06 01:17:31.514402417 +0000 UTC m=+1613.843761253"
Sep 06 01:17:38.632063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:38.632035    2575 scope.go:117] "RemoveContainer" containerID="1b442f08fa39dc6cf20e16184196f04fe28280a28993b4726d91cfd7580c2b50"
Sep 06 01:17:38.642276 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:38.642246    2575 scope.go:117] "RemoveContainer" containerID="61b4b443cd370a323f62e8dd473e0e26861190e60a080aa14402a3e5cc995bd2"
Sep 06 01:17:42.487736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:42.487706    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:17:42.488208 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:42.487908    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:17:53.182725 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:53.182695    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:17:59.980945 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:17:59.980912    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:00.004530 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.004500    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"]
Sep 06 01:18:00.004771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.004714    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt" podUID="8adcd102-0c8f-49ce-85cd-8f7921793fdd" containerName="payload-processing" containerID="cri-o://b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446" gracePeriod=30
Sep 06 01:18:00.012722 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.012696    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"]
Sep 06 01:18:00.012993 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.012968    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l" podUID="c8987733-c3cb-4280-8549-103ab312a952" containerName="payload-pre-processing" containerID="cri-o://913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404" gracePeriod=30
Sep 06 01:18:00.081385 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.081355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:00.163654 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.163615    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:00.233715 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.233608    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:00.287437 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.287410    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:18:00.290401 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.290380    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:18:00.397799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.397764    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm5dx\" (UniqueName: \"kubernetes.io/projected/8adcd102-0c8f-49ce-85cd-8f7921793fdd-kube-api-access-qm5dx\") pod \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") "
Sep 06 01:18:00.398009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.397914    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8adcd102-0c8f-49ce-85cd-8f7921793fdd-plugins-config-volume\") pod \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\" (UID: \"8adcd102-0c8f-49ce-85cd-8f7921793fdd\") "
Sep 06 01:18:00.398009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.397948    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8987733-c3cb-4280-8549-103ab312a952-plugins-config-volume\") pod \"c8987733-c3cb-4280-8549-103ab312a952\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") "
Sep 06 01:18:00.398009 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.397992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msxsf\" (UniqueName: \"kubernetes.io/projected/c8987733-c3cb-4280-8549-103ab312a952-kube-api-access-msxsf\") pod \"c8987733-c3cb-4280-8549-103ab312a952\" (UID: \"c8987733-c3cb-4280-8549-103ab312a952\") "
Sep 06 01:18:00.398302 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.398278    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8adcd102-0c8f-49ce-85cd-8f7921793fdd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8adcd102-0c8f-49ce-85cd-8f7921793fdd" (UID: "8adcd102-0c8f-49ce-85cd-8f7921793fdd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:18:00.398302 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.398287    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8987733-c3cb-4280-8549-103ab312a952-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8987733-c3cb-4280-8549-103ab312a952" (UID: "c8987733-c3cb-4280-8549-103ab312a952"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:18:00.400077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.400057    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8adcd102-0c8f-49ce-85cd-8f7921793fdd-kube-api-access-qm5dx" (OuterVolumeSpecName: "kube-api-access-qm5dx") pod "8adcd102-0c8f-49ce-85cd-8f7921793fdd" (UID: "8adcd102-0c8f-49ce-85cd-8f7921793fdd"). InnerVolumeSpecName "kube-api-access-qm5dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:18:00.400145 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.400099    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8987733-c3cb-4280-8549-103ab312a952-kube-api-access-msxsf" (OuterVolumeSpecName: "kube-api-access-msxsf") pod "c8987733-c3cb-4280-8549-103ab312a952" (UID: "c8987733-c3cb-4280-8549-103ab312a952"). InnerVolumeSpecName "kube-api-access-msxsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:18:00.499001 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.498932    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8adcd102-0c8f-49ce-85cd-8f7921793fdd-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:00.499001 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.498958    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8987733-c3cb-4280-8549-103ab312a952-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:00.499001 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.498968    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msxsf\" (UniqueName: \"kubernetes.io/projected/c8987733-c3cb-4280-8549-103ab312a952-kube-api-access-msxsf\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:00.499001 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.498977    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm5dx\" (UniqueName: \"kubernetes.io/projected/8adcd102-0c8f-49ce-85cd-8f7921793fdd-kube-api-access-qm5dx\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:00.575430 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.575401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:00.663493 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.663461    2575 generic.go:356] "Generic (PLEG): container finished" podID="8adcd102-0c8f-49ce-85cd-8f7921793fdd" containerID="b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446" exitCode=0
Sep 06 01:18:00.663645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.663507    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"
Sep 06 01:18:00.663645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.663550    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt" event={"ID":"8adcd102-0c8f-49ce-85cd-8f7921793fdd","Type":"ContainerDied","Data":"b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446"}
Sep 06 01:18:00.663645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.663600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt" event={"ID":"8adcd102-0c8f-49ce-85cd-8f7921793fdd","Type":"ContainerDied","Data":"a2f8bfc9e184492f9ec7d3dbdfc607ca073382a6eca087ac9ddb29d7a357c108"}
Sep 06 01:18:00.663645 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.663628    2575 scope.go:117] "RemoveContainer" containerID="b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446"
Sep 06 01:18:00.664939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.664914    2575 generic.go:356] "Generic (PLEG): container finished" podID="c8987733-c3cb-4280-8549-103ab312a952" containerID="913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404" exitCode=0
Sep 06 01:18:00.665049 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.664940    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l" event={"ID":"c8987733-c3cb-4280-8549-103ab312a952","Type":"ContainerDied","Data":"913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404"}
Sep 06 01:18:00.665049 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.664961    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"
Sep 06 01:18:00.665049 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.664980    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l" event={"ID":"c8987733-c3cb-4280-8549-103ab312a952","Type":"ContainerDied","Data":"d27db83b2adcbe19445e707a53e7fb04fc3b41a0f72a459b9cb4e33ec3256f5d"}
Sep 06 01:18:00.674770 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.674696    2575 scope.go:117] "RemoveContainer" containerID="b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446"
Sep 06 01:18:00.675071 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:18:00.675050    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446\": container with ID starting with b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446 not found: ID does not exist" containerID="b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446"
Sep 06 01:18:00.675154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.675081    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446"} err="failed to get container status \"b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446\": rpc error: code = NotFound desc = could not find container \"b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446\": container with ID starting with b3aafa585cf5c4b478f304e1fbdf4691543434f9f1375f4004e06f7e22b35446 not found: ID does not exist"
Sep 06 01:18:00.675154 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.675104    2575 scope.go:117] "RemoveContainer" containerID="913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404"
Sep 06 01:18:00.683798 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.683706    2575 scope.go:117] "RemoveContainer" containerID="913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404"
Sep 06 01:18:00.684081 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:18:00.684063    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404\": container with ID starting with 913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404 not found: ID does not exist" containerID="913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404"
Sep 06 01:18:00.684157 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.684088    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404"} err="failed to get container status \"913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404\": rpc error: code = NotFound desc = could not find container \"913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404\": container with ID starting with 913e9d78a64fe10252d91e70fb7eac7540c188cd172812f584748668ab53a404 not found: ID does not exist"
Sep 06 01:18:00.686342 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.686322    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"]
Sep 06 01:18:00.692242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.692221    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3032be-7c7967fd97-cj6xt"]
Sep 06 01:18:00.701695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.701674    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"]
Sep 06 01:18:00.705747 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:00.705726    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3032be-7b4f6844c8-rtw4l"]
Sep 06 01:18:02.202724 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:02.202690    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8adcd102-0c8f-49ce-85cd-8f7921793fdd" path="/var/lib/kubelet/pods/8adcd102-0c8f-49ce-85cd-8f7921793fdd/volumes"
Sep 06 01:18:02.203144 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:02.203067    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8987733-c3cb-4280-8549-103ab312a952" path="/var/lib/kubelet/pods/c8987733-c3cb-4280-8549-103ab312a952/volumes"
Sep 06 01:18:06.278226 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:06.278189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:06.345388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:06.345355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:06.577505 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:06.577416    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:10.409360 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.409321    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"]
Sep 06 01:18:10.409950 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.409894    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8adcd102-0c8f-49ce-85cd-8f7921793fdd" containerName="payload-processing"
Sep 06 01:18:10.409950 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.409915    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8adcd102-0c8f-49ce-85cd-8f7921793fdd" containerName="payload-processing"
Sep 06 01:18:10.409950 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.409941    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8987733-c3cb-4280-8549-103ab312a952" containerName="payload-pre-processing"
Sep 06 01:18:10.409950 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.409950    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8987733-c3cb-4280-8549-103ab312a952" containerName="payload-pre-processing"
Sep 06 01:18:10.410185 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.410072    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8adcd102-0c8f-49ce-85cd-8f7921793fdd" containerName="payload-processing"
Sep 06 01:18:10.410185 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.410090    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8987733-c3cb-4280-8549-103ab312a952" containerName="payload-pre-processing"
Sep 06 01:18:10.412967 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.412946    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:10.415116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.415094    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:18:10.415210 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.415100    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:18:10.420771 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.420750    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"]
Sep 06 01:18:10.589650 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.589611    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dc76cb-640d-47ff-8b23-32dc5c3181ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b\" (UID: \"38dc76cb-640d-47ff-8b23-32dc5c3181ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:10.647497 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.647468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:10.690766 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.690689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dc76cb-640d-47ff-8b23-32dc5c3181ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b\" (UID: \"38dc76cb-640d-47ff-8b23-32dc5c3181ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:10.691305 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.691286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dc76cb-640d-47ff-8b23-32dc5c3181ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b\" (UID: \"38dc76cb-640d-47ff-8b23-32dc5c3181ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:10.724979 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:10.724951    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:11.002768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:11.002744    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"]
Sep 06 01:18:11.004212 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:18:11.004189    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dc76cb_640d_47ff_8b23_32dc5c3181ae.slice/crio-a3b3f3b7c7ae8e6088a4b29cfa4dac88f89c7dc57ccb727ddd2c45b57f63e186 WatchSource:0}: Error finding container a3b3f3b7c7ae8e6088a4b29cfa4dac88f89c7dc57ccb727ddd2c45b57f63e186: Status 404 returned error can't find the container with id a3b3f3b7c7ae8e6088a4b29cfa4dac88f89c7dc57ccb727ddd2c45b57f63e186
Sep 06 01:18:11.734997 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:11.734966    2575 generic.go:356] "Generic (PLEG): container finished" podID="38dc76cb-640d-47ff-8b23-32dc5c3181ae" containerID="cd68192b10d7d3625665906c80e5c1c31fb99dcb4253cb6ce2a904c71f01b1c4" exitCode=0
Sep 06 01:18:11.735378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:11.735056    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b" event={"ID":"38dc76cb-640d-47ff-8b23-32dc5c3181ae","Type":"ContainerDied","Data":"cd68192b10d7d3625665906c80e5c1c31fb99dcb4253cb6ce2a904c71f01b1c4"}
Sep 06 01:18:11.735378 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:11.735105    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b" event={"ID":"38dc76cb-640d-47ff-8b23-32dc5c3181ae","Type":"ContainerStarted","Data":"a3b3f3b7c7ae8e6088a4b29cfa4dac88f89c7dc57ccb727ddd2c45b57f63e186"}
Sep 06 01:18:12.869836 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:12.869813    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:13.011537 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:13.011451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dc76cb-640d-47ff-8b23-32dc5c3181ae-maas-api-service-ca\") pod \"38dc76cb-640d-47ff-8b23-32dc5c3181ae\" (UID: \"38dc76cb-640d-47ff-8b23-32dc5c3181ae\") "
Sep 06 01:18:13.011776 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:13.011756    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38dc76cb-640d-47ff-8b23-32dc5c3181ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38dc76cb-640d-47ff-8b23-32dc5c3181ae" (UID: "38dc76cb-640d-47ff-8b23-32dc5c3181ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:18:13.113045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:13.113014    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dc76cb-640d-47ff-8b23-32dc5c3181ae-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:13.749052 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:13.749022    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"
Sep 06 01:18:13.749244 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:13.749054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b" event={"ID":"38dc76cb-640d-47ff-8b23-32dc5c3181ae","Type":"ContainerDied","Data":"a3b3f3b7c7ae8e6088a4b29cfa4dac88f89c7dc57ccb727ddd2c45b57f63e186"}
Sep 06 01:18:13.749244 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:13.749103    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3b3f3b7c7ae8e6088a4b29cfa4dac88f89c7dc57ccb727ddd2c45b57f63e186"
Sep 06 01:18:25.968576 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:25.968540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:25.994839 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:25.994808    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"]
Sep 06 01:18:25.995106 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:25.995078    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c" podUID="e40dfedf-864c-41cb-8a72-5ad7de2908fb" containerName="payload-processing" containerID="cri-o://a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355" gracePeriod=30
Sep 06 01:18:26.044543 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.044502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:26.106555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.106517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:26.163376 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.163347    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"]
Sep 06 01:18:26.167237 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.167207    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b2e4f6-x2l9b"]
Sep 06 01:18:26.170910 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.170877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:26.202254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.202230    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38dc76cb-640d-47ff-8b23-32dc5c3181ae" path="/var/lib/kubelet/pods/38dc76cb-640d-47ff-8b23-32dc5c3181ae/volumes"
Sep 06 01:18:26.231695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.231615    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:26.256388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.256367    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:18:26.325141 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.325106    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40dfedf-864c-41cb-8a72-5ad7de2908fb-plugins-config-volume\") pod \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") "
Sep 06 01:18:26.325323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.325193    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tllpd\" (UniqueName: \"kubernetes.io/projected/e40dfedf-864c-41cb-8a72-5ad7de2908fb-kube-api-access-tllpd\") pod \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\" (UID: \"e40dfedf-864c-41cb-8a72-5ad7de2908fb\") "
Sep 06 01:18:26.325530 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.325505    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e40dfedf-864c-41cb-8a72-5ad7de2908fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e40dfedf-864c-41cb-8a72-5ad7de2908fb" (UID: "e40dfedf-864c-41cb-8a72-5ad7de2908fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:18:26.328244 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.327998    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e40dfedf-864c-41cb-8a72-5ad7de2908fb-kube-api-access-tllpd" (OuterVolumeSpecName: "kube-api-access-tllpd") pod "e40dfedf-864c-41cb-8a72-5ad7de2908fb" (UID: "e40dfedf-864c-41cb-8a72-5ad7de2908fb"). InnerVolumeSpecName "kube-api-access-tllpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:18:26.426834 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.426800    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40dfedf-864c-41cb-8a72-5ad7de2908fb-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:26.426834 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.426832    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tllpd\" (UniqueName: \"kubernetes.io/projected/e40dfedf-864c-41cb-8a72-5ad7de2908fb-kube-api-access-tllpd\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:26.544975 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.544885    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:26.824074 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.823980    2575 generic.go:356] "Generic (PLEG): container finished" podID="e40dfedf-864c-41cb-8a72-5ad7de2908fb" containerID="a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355" exitCode=0
Sep 06 01:18:26.824074 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.824025    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"
Sep 06 01:18:26.824074 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.824046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c" event={"ID":"e40dfedf-864c-41cb-8a72-5ad7de2908fb","Type":"ContainerDied","Data":"a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355"}
Sep 06 01:18:26.824323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.824087    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c" event={"ID":"e40dfedf-864c-41cb-8a72-5ad7de2908fb","Type":"ContainerDied","Data":"6acbf7f1d19f78d3193437f8fe71523ac468ebf6e301a39eb61737790226ac74"}
Sep 06 01:18:26.824323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.824109    2575 scope.go:117] "RemoveContainer" containerID="a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355"
Sep 06 01:18:26.833908 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.833887    2575 scope.go:117] "RemoveContainer" containerID="a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355"
Sep 06 01:18:26.834192 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:18:26.834173    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355\": container with ID starting with a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355 not found: ID does not exist" containerID="a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355"
Sep 06 01:18:26.834246 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.834200    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355"} err="failed to get container status \"a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355\": rpc error: code = NotFound desc = could not find container \"a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355\": container with ID starting with a4f9287e1469f16d45214ee33be849d108d8fdf892e30a2ff19827488de8c355 not found: ID does not exist"
Sep 06 01:18:26.845357 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.845335    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"]
Sep 06 01:18:26.849620 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:26.849592    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b2e4f6-7d899bb7bc-bt88c"]
Sep 06 01:18:28.202383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:28.202346    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e40dfedf-864c-41cb-8a72-5ad7de2908fb" path="/var/lib/kubelet/pods/e40dfedf-864c-41cb-8a72-5ad7de2908fb/volumes"
Sep 06 01:18:31.528458 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:31.528420    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"]
Sep 06 01:18:31.577192 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:31.577158    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:18:46.548736 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.548668    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m" podUID="0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" containerName="main" containerID="cri-o://d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4" gracePeriod=45
Sep 06 01:18:46.803415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.803356    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:18:46.915117 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915085    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wptbw\" (UniqueName: \"kubernetes.io/projected/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-kube-api-access-wptbw\") pod \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") "
Sep 06 01:18:46.915274 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915145    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-model-cache\") pod \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") "
Sep 06 01:18:46.915274 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915164    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tls-certs\") pod \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") "
Sep 06 01:18:46.915352 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915328    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tmp-dir\") pod \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") "
Sep 06 01:18:46.915416 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915400    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-dshm\") pod \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") "
Sep 06 01:18:46.915459 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915408    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" (UID: "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:18:46.915517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915469    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-home\") pod \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\" (UID: \"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c\") "
Sep 06 01:18:46.915573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915534    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" (UID: "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:18:46.915763 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915737    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-home" (OuterVolumeSpecName: "home") pod "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" (UID: "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:18:46.916075 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915776    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:46.916075 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.915789    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:46.917518 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.917496    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-dshm" (OuterVolumeSpecName: "dshm") pod "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" (UID: "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:18:46.917611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.917529    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-kube-api-access-wptbw" (OuterVolumeSpecName: "kube-api-access-wptbw") pod "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" (UID: "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c"). InnerVolumeSpecName "kube-api-access-wptbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:18:46.917611 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.917594    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" (UID: "0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:18:46.948119 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.948095    2575 generic.go:356] "Generic (PLEG): container finished" podID="0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" containerID="d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4" exitCode=0
Sep 06 01:18:46.948247 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.948145    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"
Sep 06 01:18:46.948247 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.948155    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m" event={"ID":"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c","Type":"ContainerDied","Data":"d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4"}
Sep 06 01:18:46.948247 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.948193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m" event={"ID":"0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c","Type":"ContainerDied","Data":"6d0958084c0137af3adfdbfc4bcd743a384fd1d3781233221e81bf08032b5c42"}
Sep 06 01:18:46.948247 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.948210    2575 scope.go:117] "RemoveContainer" containerID="d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4"
Sep 06 01:18:46.958260 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.958240    2575 scope.go:117] "RemoveContainer" containerID="d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4"
Sep 06 01:18:46.958553 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:18:46.958529    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4\": container with ID starting with d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4 not found: ID does not exist" containerID="d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4"
Sep 06 01:18:46.958650 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.958558    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4"} err="failed to get container status \"d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4\": rpc error: code = NotFound desc = could not find container \"d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4\": container with ID starting with d17864653dc3106f7c7af7adfb2c773a1a39fee2e7d5f4404827ce774fdfe2a4 not found: ID does not exist"
Sep 06 01:18:46.977452 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.977426    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"]
Sep 06 01:18:46.980991 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:46.980966    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-b2e4f6/ipp-route-b2e4f6-kserve-77987654f5-pcz8m"]
Sep 06 01:18:47.017016 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:47.016992    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:47.017016 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:47.017015    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:47.017116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:47.017023    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:47.017116 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:47.017031    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wptbw\" (UniqueName: \"kubernetes.io/projected/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c-kube-api-access-wptbw\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:18:48.201737 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:18:48.201703    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" path="/var/lib/kubelet/pods/0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c/volumes"
Sep 06 01:19:00.632340 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:00.632257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:08.074491 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:08.074457    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:08.624481 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:08.624442    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:14.097695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:14.097655    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:19.791103 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:19.791066    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:19.891056 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:19.891030    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:20.075115 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.075032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:20.459427 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459396    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"]
Sep 06 01:19:20.459764 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459752    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" containerName="main"
Sep 06 01:19:20.459808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459766    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" containerName="main"
Sep 06 01:19:20.459808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459773    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38dc76cb-640d-47ff-8b23-32dc5c3181ae" containerName="revoke-keys"
Sep 06 01:19:20.459808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459778    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="38dc76cb-640d-47ff-8b23-32dc5c3181ae" containerName="revoke-keys"
Sep 06 01:19:20.459808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459790    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e40dfedf-864c-41cb-8a72-5ad7de2908fb" containerName="payload-processing"
Sep 06 01:19:20.459808 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459795    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e40dfedf-864c-41cb-8a72-5ad7de2908fb" containerName="payload-processing"
Sep 06 01:19:20.460019 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459872    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="38dc76cb-640d-47ff-8b23-32dc5c3181ae" containerName="revoke-keys"
Sep 06 01:19:20.460019 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459884    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e40dfedf-864c-41cb-8a72-5ad7de2908fb" containerName="payload-processing"
Sep 06 01:19:20.460019 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.459896    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d405c8b-2f60-4d0c-b669-f3d6c8f0cc8c" containerName="main"
Sep 06 01:19:20.463013 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.462997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:20.465563 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.465544    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:19:20.465750 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.465735    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:19:20.472672 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.472653    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"]
Sep 06 01:19:20.518025 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.518002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbfbe020-22e4-4c2b-83a8-08378f166111-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf\" (UID: \"bbfbe020-22e4-4c2b-83a8-08378f166111\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:20.556479 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.556455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:20.618501 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.618471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbfbe020-22e4-4c2b-83a8-08378f166111-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf\" (UID: \"bbfbe020-22e4-4c2b-83a8-08378f166111\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:20.619114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.619094    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbfbe020-22e4-4c2b-83a8-08378f166111-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf\" (UID: \"bbfbe020-22e4-4c2b-83a8-08378f166111\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:20.774092 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:20.774010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:21.047108 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:21.047078    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"]
Sep 06 01:19:21.048708 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:19:21.048680    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfbe020_22e4_4c2b_83a8_08378f166111.slice/crio-9d475450e3081332ec0f0499f88605ea38d89021a197f59642c017596106a3fc WatchSource:0}: Error finding container 9d475450e3081332ec0f0499f88605ea38d89021a197f59642c017596106a3fc: Status 404 returned error can't find the container with id 9d475450e3081332ec0f0499f88605ea38d89021a197f59642c017596106a3fc
Sep 06 01:19:21.148174 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:21.148142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf" event={"ID":"bbfbe020-22e4-4c2b-83a8-08378f166111","Type":"ContainerStarted","Data":"9d475450e3081332ec0f0499f88605ea38d89021a197f59642c017596106a3fc"}
Sep 06 01:19:22.155322 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:22.155238    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbfbe020-22e4-4c2b-83a8-08378f166111" containerID="e01ab4e1d1802dc0fc84b0fb9c4f651cda05ac903e80ca1772f22ea53cc683c8" exitCode=0
Sep 06 01:19:22.155661 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:22.155318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf" event={"ID":"bbfbe020-22e4-4c2b-83a8-08378f166111","Type":"ContainerDied","Data":"e01ab4e1d1802dc0fc84b0fb9c4f651cda05ac903e80ca1772f22ea53cc683c8"}
Sep 06 01:19:23.284385 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:23.284358    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:23.342510 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:23.342477    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbfbe020-22e4-4c2b-83a8-08378f166111-maas-api-service-ca\") pod \"bbfbe020-22e4-4c2b-83a8-08378f166111\" (UID: \"bbfbe020-22e4-4c2b-83a8-08378f166111\") "
Sep 06 01:19:23.342841 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:23.342818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbfbe020-22e4-4c2b-83a8-08378f166111-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbfbe020-22e4-4c2b-83a8-08378f166111" (UID: "bbfbe020-22e4-4c2b-83a8-08378f166111"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:19:23.443983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:23.443916    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbfbe020-22e4-4c2b-83a8-08378f166111-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:24.168386 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:24.168357    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"
Sep 06 01:19:24.168565 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:24.168402    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf" event={"ID":"bbfbe020-22e4-4c2b-83a8-08378f166111","Type":"ContainerDied","Data":"9d475450e3081332ec0f0499f88605ea38d89021a197f59642c017596106a3fc"}
Sep 06 01:19:24.168565 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:24.168448    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d475450e3081332ec0f0499f88605ea38d89021a197f59642c017596106a3fc"
Sep 06 01:19:30.956473 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:30.956442    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:30.984772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:30.984736    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"]
Sep 06 01:19:30.985033 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:30.985008    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg" podUID="1e4ec456-04a8-4a07-bc8d-1623930eed24" containerName="payload-processing" containerID="cri-o://1320e832015493c9a1453597b0f3e173c71f5842b33428cf10eb5a73c2112b66" gracePeriod=30
Sep 06 01:19:30.990600 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:30.990568    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"]
Sep 06 01:19:30.990810 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:30.990784    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4" podUID="50939f74-fa59-45d0-84f0-4327f4623b0a" containerName="payload-pre-processing" containerID="cri-o://55a2685de9aa71a6437629a689edd3099c148c5743500144613e7881e221f227" gracePeriod=30
Sep 06 01:19:31.056778 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.056751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:31.152305 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.152024    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"]
Sep 06 01:19:31.154259 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.153836    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:31.156544 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.156521    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-074d50-b9zzf"]
Sep 06 01:19:31.213047 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.212974    2575 generic.go:356] "Generic (PLEG): container finished" podID="1e4ec456-04a8-4a07-bc8d-1623930eed24" containerID="1320e832015493c9a1453597b0f3e173c71f5842b33428cf10eb5a73c2112b66" exitCode=0
Sep 06 01:19:31.213205 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.213048    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg" event={"ID":"1e4ec456-04a8-4a07-bc8d-1623930eed24","Type":"ContainerDied","Data":"1320e832015493c9a1453597b0f3e173c71f5842b33428cf10eb5a73c2112b66"}
Sep 06 01:19:31.215148 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.215124    2575 generic.go:356] "Generic (PLEG): container finished" podID="50939f74-fa59-45d0-84f0-4327f4623b0a" containerID="55a2685de9aa71a6437629a689edd3099c148c5743500144613e7881e221f227" exitCode=0
Sep 06 01:19:31.215274 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.215194    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4" event={"ID":"50939f74-fa59-45d0-84f0-4327f4623b0a","Type":"ContainerDied","Data":"55a2685de9aa71a6437629a689edd3099c148c5743500144613e7881e221f227"}
Sep 06 01:19:31.222482 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.222462    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:31.261836 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.261815    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:19:31.265240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.265215    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:19:31.315402 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.315370    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgzpf\" (UniqueName: \"kubernetes.io/projected/1e4ec456-04a8-4a07-bc8d-1623930eed24-kube-api-access-zgzpf\") pod \"1e4ec456-04a8-4a07-bc8d-1623930eed24\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") "
Sep 06 01:19:31.315572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.315452    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec456-04a8-4a07-bc8d-1623930eed24-plugins-config-volume\") pod \"1e4ec456-04a8-4a07-bc8d-1623930eed24\" (UID: \"1e4ec456-04a8-4a07-bc8d-1623930eed24\") "
Sep 06 01:19:31.315572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.315490    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50939f74-fa59-45d0-84f0-4327f4623b0a-plugins-config-volume\") pod \"50939f74-fa59-45d0-84f0-4327f4623b0a\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") "
Sep 06 01:19:31.315572 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.315537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76wkb\" (UniqueName: \"kubernetes.io/projected/50939f74-fa59-45d0-84f0-4327f4623b0a-kube-api-access-76wkb\") pod \"50939f74-fa59-45d0-84f0-4327f4623b0a\" (UID: \"50939f74-fa59-45d0-84f0-4327f4623b0a\") "
Sep 06 01:19:31.315867 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.315820    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e4ec456-04a8-4a07-bc8d-1623930eed24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e4ec456-04a8-4a07-bc8d-1623930eed24" (UID: "1e4ec456-04a8-4a07-bc8d-1623930eed24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:19:31.316157 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.316115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50939f74-fa59-45d0-84f0-4327f4623b0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50939f74-fa59-45d0-84f0-4327f4623b0a" (UID: "50939f74-fa59-45d0-84f0-4327f4623b0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:19:31.318890 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.318847    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50939f74-fa59-45d0-84f0-4327f4623b0a-kube-api-access-76wkb" (OuterVolumeSpecName: "kube-api-access-76wkb") pod "50939f74-fa59-45d0-84f0-4327f4623b0a" (UID: "50939f74-fa59-45d0-84f0-4327f4623b0a"). InnerVolumeSpecName "kube-api-access-76wkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:19:31.319061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.319037    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e4ec456-04a8-4a07-bc8d-1623930eed24-kube-api-access-zgzpf" (OuterVolumeSpecName: "kube-api-access-zgzpf") pod "1e4ec456-04a8-4a07-bc8d-1623930eed24" (UID: "1e4ec456-04a8-4a07-bc8d-1623930eed24"). InnerVolumeSpecName "kube-api-access-zgzpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:19:31.417058 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.417026    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgzpf\" (UniqueName: \"kubernetes.io/projected/1e4ec456-04a8-4a07-bc8d-1623930eed24-kube-api-access-zgzpf\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:31.417058 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.417052    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4ec456-04a8-4a07-bc8d-1623930eed24-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:31.417058 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.417061    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50939f74-fa59-45d0-84f0-4327f4623b0a-plugins-config-volume\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:31.417279 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.417069    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76wkb\" (UniqueName: \"kubernetes.io/projected/50939f74-fa59-45d0-84f0-4327f4623b0a-kube-api-access-76wkb\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:31.443949 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.443919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:31.544747 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.544670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:31.643962 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:31.643932    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:32.202016 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.201987    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbfbe020-22e4-4c2b-83a8-08378f166111" path="/var/lib/kubelet/pods/bbfbe020-22e4-4c2b-83a8-08378f166111/volumes"
Sep 06 01:19:32.222099 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.222071    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"
Sep 06 01:19:32.222214 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.222094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg" event={"ID":"1e4ec456-04a8-4a07-bc8d-1623930eed24","Type":"ContainerDied","Data":"cabe09df3812b68cd0c2b947248cadac20a3dd63509a1ee26f0d43dc0747ff9e"}
Sep 06 01:19:32.222214 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.222163    2575 scope.go:117] "RemoveContainer" containerID="1320e832015493c9a1453597b0f3e173c71f5842b33428cf10eb5a73c2112b66"
Sep 06 01:19:32.223565 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.223544    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"
Sep 06 01:19:32.223687 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.223565    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4" event={"ID":"50939f74-fa59-45d0-84f0-4327f4623b0a","Type":"ContainerDied","Data":"b799cffd2055b2c5f1fd1e717da1aa737b4e19347848634ca0cfc1d607a6f999"}
Sep 06 01:19:32.231750 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.231722    2575 scope.go:117] "RemoveContainer" containerID="55a2685de9aa71a6437629a689edd3099c148c5743500144613e7881e221f227"
Sep 06 01:19:32.241931 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.241906    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"]
Sep 06 01:19:32.247627 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.247600    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-074d50-68bf79684b-zq6c4"]
Sep 06 01:19:32.256339 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.256318    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"]
Sep 06 01:19:32.260114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:32.260095    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-074d50-5fbd896fb7-vd9jg"]
Sep 06 01:19:34.201911 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:34.201840    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e4ec456-04a8-4a07-bc8d-1623930eed24" path="/var/lib/kubelet/pods/1e4ec456-04a8-4a07-bc8d-1623930eed24/volumes"
Sep 06 01:19:34.202447 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:34.202427    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50939f74-fa59-45d0-84f0-4327f4623b0a" path="/var/lib/kubelet/pods/50939f74-fa59-45d0-84f0-4327f4623b0a/volumes"
Sep 06 01:19:37.257728 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.257690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:37.320926 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.320898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:37.507921 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.507818    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"]
Sep 06 01:19:37.508077 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.508054    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy" containerID="cri-o://0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20" gracePeriod=30
Sep 06 01:19:37.528145 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.528117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:37.894325 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894229    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"]
Sep 06 01:19:37.894668 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894649    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50939f74-fa59-45d0-84f0-4327f4623b0a" containerName="payload-pre-processing"
Sep 06 01:19:37.894748 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894671    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="50939f74-fa59-45d0-84f0-4327f4623b0a" containerName="payload-pre-processing"
Sep 06 01:19:37.894748 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894683    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbfbe020-22e4-4c2b-83a8-08378f166111" containerName="revoke-keys"
Sep 06 01:19:37.894748 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894691    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbfbe020-22e4-4c2b-83a8-08378f166111" containerName="revoke-keys"
Sep 06 01:19:37.894748 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894728    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4ec456-04a8-4a07-bc8d-1623930eed24" containerName="payload-processing"
Sep 06 01:19:37.894748 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894737    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4ec456-04a8-4a07-bc8d-1623930eed24" containerName="payload-processing"
Sep 06 01:19:37.895050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894831    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="50939f74-fa59-45d0-84f0-4327f4623b0a" containerName="payload-pre-processing"
Sep 06 01:19:37.895050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894845    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4ec456-04a8-4a07-bc8d-1623930eed24" containerName="payload-processing"
Sep 06 01:19:37.895050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.894876    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbfbe020-22e4-4c2b-83a8-08378f166111" containerName="revoke-keys"
Sep 06 01:19:37.899553 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.899532    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:37.902093 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.902069    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:19:37.902584 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.902558    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:19:37.905568 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.905546    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"]
Sep 06 01:19:37.977939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:37.977909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/524548d1-1e13-4ba5-b21b-46c807038ef9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss\" (UID: \"524548d1-1e13-4ba5-b21b-46c807038ef9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:38.079384 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.079347    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/524548d1-1e13-4ba5-b21b-46c807038ef9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss\" (UID: \"524548d1-1e13-4ba5-b21b-46c807038ef9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:38.079974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.079956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/524548d1-1e13-4ba5-b21b-46c807038ef9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss\" (UID: \"524548d1-1e13-4ba5-b21b-46c807038ef9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:38.120208 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.120177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:38.213823 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.213794    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:19:38.222030 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.222009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:38.494544 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.494516    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"]
Sep 06 01:19:38.496271 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:19:38.496245    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524548d1_1e13_4ba5_b21b_46c807038ef9.slice/crio-333e23f49ff9cb841934bfdf97775478d193d6a4969af4ac504c38281e732d08 WatchSource:0}: Error finding container 333e23f49ff9cb841934bfdf97775478d193d6a4969af4ac504c38281e732d08: Status 404 returned error can't find the container with id 333e23f49ff9cb841934bfdf97775478d193d6a4969af4ac504c38281e732d08
Sep 06 01:19:38.689248 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:38.689226    2575 scope.go:117] "RemoveContainer" containerID="065fa3a6b33f5bd3a67f6a4073b89fa6b228072748f3987f48b55890a24acd9e"
Sep 06 01:19:39.266939 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:39.266906    2575 generic.go:356] "Generic (PLEG): container finished" podID="524548d1-1e13-4ba5-b21b-46c807038ef9" containerID="870eea5a80376a0c4ab8b7fef5a835375aed157f3be39190a6525153ab5ef714" exitCode=0
Sep 06 01:19:39.267113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:39.267003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss" event={"ID":"524548d1-1e13-4ba5-b21b-46c807038ef9","Type":"ContainerDied","Data":"870eea5a80376a0c4ab8b7fef5a835375aed157f3be39190a6525153ab5ef714"}
Sep 06 01:19:39.267113 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:39.267054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss" event={"ID":"524548d1-1e13-4ba5-b21b-46c807038ef9","Type":"ContainerStarted","Data":"333e23f49ff9cb841934bfdf97775478d193d6a4969af4ac504c38281e732d08"}
Sep 06 01:19:40.402492 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:40.402468    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:40.502392 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:40.502345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/524548d1-1e13-4ba5-b21b-46c807038ef9-maas-api-service-ca\") pod \"524548d1-1e13-4ba5-b21b-46c807038ef9\" (UID: \"524548d1-1e13-4ba5-b21b-46c807038ef9\") "
Sep 06 01:19:40.502681 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:40.502659    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524548d1-1e13-4ba5-b21b-46c807038ef9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "524548d1-1e13-4ba5-b21b-46c807038ef9" (UID: "524548d1-1e13-4ba5-b21b-46c807038ef9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:19:40.603730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:40.603647    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/524548d1-1e13-4ba5-b21b-46c807038ef9-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:41.278516 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:41.278490    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"
Sep 06 01:19:41.278691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:41.278538    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss" event={"ID":"524548d1-1e13-4ba5-b21b-46c807038ef9","Type":"ContainerDied","Data":"333e23f49ff9cb841934bfdf97775478d193d6a4969af4ac504c38281e732d08"}
Sep 06 01:19:41.278691 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:41.278580    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="333e23f49ff9cb841934bfdf97775478d193d6a4969af4ac504c38281e732d08"
Sep 06 01:19:42.753492 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.753468    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:19:42.927983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.927906    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-certs\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.927983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.927943    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-data\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.927983 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.927965    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-credential-socket\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928021    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-socket\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928071    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d7jr\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-kube-api-access-5d7jr\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928105    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-envoy\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928127    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-token\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e7f7f-834a-402c-9d96-295ac56f2d20-istiod-ca-cert\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928230 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928214    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-podinfo\") pod \"347e7f7f-834a-402c-9d96-295ac56f2d20\" (UID: \"347e7f7f-834a-402c-9d96-295ac56f2d20\") "
Sep 06 01:19:42.928522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928238    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:19:42.928522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928253    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-data" (OuterVolumeSpecName: "istio-data") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:19:42.928522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928321    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:19:42.928522 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928336    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:19:42.928742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928576    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:42.928742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928594    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:42.928742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928610    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:42.928742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928638    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:42.928742 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.928610    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347e7f7f-834a-402c-9d96-295ac56f2d20-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:19:42.930491 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.930466    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:19:42.930598 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.930583    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:19:42.930899 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.930868    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-token" (OuterVolumeSpecName: "istio-token") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:19:42.931003 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:42.930873    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-kube-api-access-5d7jr" (OuterVolumeSpecName: "kube-api-access-5d7jr") pod "347e7f7f-834a-402c-9d96-295ac56f2d20" (UID: "347e7f7f-834a-402c-9d96-295ac56f2d20"). InnerVolumeSpecName "kube-api-access-5d7jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:19:43.029714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.029684    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d7jr\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-kube-api-access-5d7jr\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:43.029714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.029713    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:43.029904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.029728    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:43.029904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.029738    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/347e7f7f-834a-402c-9d96-295ac56f2d20-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:43.029904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.029749    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/347e7f7f-834a-402c-9d96-295ac56f2d20-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:43.291106 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.291074    2575 generic.go:356] "Generic (PLEG): container finished" podID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerID="0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20" exitCode=0
Sep 06 01:19:43.291306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.291127    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"
Sep 06 01:19:43.291306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.291142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" event={"ID":"347e7f7f-834a-402c-9d96-295ac56f2d20","Type":"ContainerDied","Data":"0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20"}
Sep 06 01:19:43.291306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.291185    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc" event={"ID":"347e7f7f-834a-402c-9d96-295ac56f2d20","Type":"ContainerDied","Data":"2aa16bb2cc057d0a309b01dd24e6466865c001478ff0ff36b031cfbfabcc5f78"}
Sep 06 01:19:43.291306 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.291206    2575 scope.go:117] "RemoveContainer" containerID="0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20"
Sep 06 01:19:43.301765 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.301744    2575 scope.go:117] "RemoveContainer" containerID="6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"
Sep 06 01:19:43.310073 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.310049    2575 scope.go:117] "RemoveContainer" containerID="0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20"
Sep 06 01:19:43.310329 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:19:43.310311    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20\": container with ID starting with 0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20 not found: ID does not exist" containerID="0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20"
Sep 06 01:19:43.310375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.310338    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20"} err="failed to get container status \"0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20\": rpc error: code = NotFound desc = could not find container \"0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20\": container with ID starting with 0f505da89725859e400edacdaa8bc1d80bf538f2b783be73b23db1f73525ae20 not found: ID does not exist"
Sep 06 01:19:43.310375 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.310356    2575 scope.go:117] "RemoveContainer" containerID="6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"
Sep 06 01:19:43.310599 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:19:43.310581    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b\": container with ID starting with 6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b not found: ID does not exist" containerID="6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"
Sep 06 01:19:43.310632 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.310605    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b"} err="failed to get container status \"6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b\": rpc error: code = NotFound desc = could not find container \"6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b\": container with ID starting with 6a88a554c2d922bb58bf8cf16e6332bac572978811b214d2d8f0ef103c272e5b not found: ID does not exist"
Sep 06 01:19:43.314268 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.314247    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"]
Sep 06 01:19:43.318887 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:43.318840    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-074d50-openshift-default-6cffd6978d-gvtfc"]
Sep 06 01:19:44.202117 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:44.202079    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" path="/var/lib/kubelet/pods/347e7f7f-834a-402c-9d96-295ac56f2d20/volumes"
Sep 06 01:19:48.413901 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.413838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:48.490659 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.490628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:48.602772 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.602737    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:48.705828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.705800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:48.716021 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.715998    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"]
Sep 06 01:19:48.719750 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.719730    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5844a8-cq7ss"]
Sep 06 01:19:48.772602 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:48.772574    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:49.128353 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:49.128266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:50.201865 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:50.201811    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="524548d1-1e13-4ba5-b21b-46c807038ef9" path="/var/lib/kubelet/pods/524548d1-1e13-4ba5-b21b-46c807038ef9/volumes"
Sep 06 01:19:55.358710 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:55.358673    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:55.459147 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:55.459116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:55.625370 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:55.625282    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:56.004585 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004549    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"]
Sep 06 01:19:56.004919 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004907    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy"
Sep 06 01:19:56.004974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004921    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy"
Sep 06 01:19:56.004974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004934    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="524548d1-1e13-4ba5-b21b-46c807038ef9" containerName="revoke-keys"
Sep 06 01:19:56.004974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004939    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="524548d1-1e13-4ba5-b21b-46c807038ef9" containerName="revoke-keys"
Sep 06 01:19:56.004974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004951    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy"
Sep 06 01:19:56.004974 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.004956    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy"
Sep 06 01:19:56.005131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.005028    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="524548d1-1e13-4ba5-b21b-46c807038ef9" containerName="revoke-keys"
Sep 06 01:19:56.005131 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.005036    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy"
Sep 06 01:19:56.009599 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.009582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:56.012433 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.012411    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xjq68\""
Sep 06 01:19:56.016100 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.016071    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:19:56.025827 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.025795    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"]
Sep 06 01:19:56.041487 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.041458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5\" (UID: \"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:56.141917 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.141848    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5\" (UID: \"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:56.142529 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.142506    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5\" (UID: \"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:56.158356 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.158330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:19:56.321223 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.321147    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:56.591979 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:56.591955    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"]
Sep 06 01:19:56.593891 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:19:56.593865    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod069a72f0_32a8_45b9_ad6a_e7a30cb6eb6b.slice/crio-07a530a5d05eb2fe174baed0351a7ac52d6586bf48579b40cea08f75910e89cd WatchSource:0}: Error finding container 07a530a5d05eb2fe174baed0351a7ac52d6586bf48579b40cea08f75910e89cd: Status 404 returned error can't find the container with id 07a530a5d05eb2fe174baed0351a7ac52d6586bf48579b40cea08f75910e89cd
Sep 06 01:19:57.372773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:57.372746    2575 generic.go:356] "Generic (PLEG): container finished" podID="069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b" containerID="70aa89164588b869e747f7af58fb0adae72d7fd22cb8741963f376bf30d5369d" exitCode=0
Sep 06 01:19:57.372932 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:57.372837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5" event={"ID":"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b","Type":"ContainerDied","Data":"70aa89164588b869e747f7af58fb0adae72d7fd22cb8741963f376bf30d5369d"}
Sep 06 01:19:57.372932 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:57.372915    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5" event={"ID":"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b","Type":"ContainerStarted","Data":"07a530a5d05eb2fe174baed0351a7ac52d6586bf48579b40cea08f75910e89cd"}
Sep 06 01:19:58.506573 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:58.506547    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:58.561090 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:58.561066    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b-maas-api-service-ca\") pod \"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b\" (UID: \"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b\") "
Sep 06 01:19:58.561388 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:58.561366    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b" (UID: "069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:19:58.661651 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:58.661582    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b-maas-api-service-ca\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:19:59.385749 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:59.385718    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5" event={"ID":"069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b","Type":"ContainerDied","Data":"07a530a5d05eb2fe174baed0351a7ac52d6586bf48579b40cea08f75910e89cd"}
Sep 06 01:19:59.385947 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:59.385756    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"
Sep 06 01:19:59.385947 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:19:59.385762    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07a530a5d05eb2fe174baed0351a7ac52d6586bf48579b40cea08f75910e89cd"
Sep 06 01:20:06.446800 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.446763    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"]
Sep 06 01:20:06.451020 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.450995    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-470346-29810955-tjbhm"]
Sep 06 01:20:06.453606 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.453580    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:06.526441 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.526387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:06.627149 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.627118    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:06.647174 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.647144    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"]
Sep 06 01:20:06.649322 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.649304    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-470346-dvqx5"]
Sep 06 01:20:06.724646 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:06.724455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:07.037570 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:07.037478    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:08.201995 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:08.201956    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b" path="/var/lib/kubelet/pods/069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b/volumes"
Sep 06 01:20:08.202415 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:08.202364    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa0aec3-8023-4e86-b399-80cd19e2f9fe" path="/var/lib/kubelet/pods/aaa0aec3-8023-4e86-b399-80cd19e2f9fe/volumes"
Sep 06 01:20:12.976183 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:12.976143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:13.077319 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:13.077291    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:13.220822 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:13.220786    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"]
Sep 06 01:20:13.221082 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:13.221056    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" containerID="cri-o://37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03" gracePeriod=30
Sep 06 01:20:13.237503 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:13.237435    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:20:17.089825 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:17.089779    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 01:20:17.090216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:17.089882    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 01:20:18.468107 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.468082    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:20:18.496052 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.495981    2575 generic.go:356] "Generic (PLEG): container finished" podID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerID="37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03" exitCode=0
Sep 06 01:20:18.496052 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.496032    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"
Sep 06 01:20:18.496229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.496061    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" event={"ID":"e58fe251-fa84-41b4-8b9b-97dd865d366d","Type":"ContainerDied","Data":"37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03"}
Sep 06 01:20:18.496229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.496110    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r" event={"ID":"e58fe251-fa84-41b4-8b9b-97dd865d366d","Type":"ContainerDied","Data":"c03844c51aa7289a36b1582cd112ad2dd6567a65ec8195fe2c80e46b63d36643"}
Sep 06 01:20:18.496229 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.496136    2575 scope.go:117] "RemoveContainer" containerID="37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03"
Sep 06 01:20:18.512407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.512381    2575 scope.go:117] "RemoveContainer" containerID="3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"
Sep 06 01:20:18.520802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.520772    2575 scope.go:117] "RemoveContainer" containerID="37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03"
Sep 06 01:20:18.521102 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:20:18.521082    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03\": container with ID starting with 37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03 not found: ID does not exist" containerID="37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03"
Sep 06 01:20:18.521232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.521110    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03"} err="failed to get container status \"37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03\": rpc error: code = NotFound desc = could not find container \"37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03\": container with ID starting with 37e95cea36f08ab6aca03246b6bf87f20d07d8deb3341666573929fc381bbb03 not found: ID does not exist"
Sep 06 01:20:18.521232 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.521127    2575 scope.go:117] "RemoveContainer" containerID="3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"
Sep 06 01:20:18.521369 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:20:18.521353    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161\": container with ID starting with 3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161 not found: ID does not exist" containerID="3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"
Sep 06 01:20:18.521406 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.521375    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161"} err="failed to get container status \"3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161\": rpc error: code = NotFound desc = could not find container \"3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161\": container with ID starting with 3d0bfb6ccb580c18e145c1a59561ba7b2b972faa5aa54e10945a929013d53161 not found: ID does not exist"
Sep 06 01:20:18.535893 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.535872    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-data\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.535981 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.535941    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcn58\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-kube-api-access-wcn58\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.535981 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.535973    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-certs\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536087 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.535994    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-podinfo\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536087 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536015    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-socket\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536087 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536035    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-credential-socket\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536087 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536077    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-token\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536303 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536105    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe251-fa84-41b4-8b9b-97dd865d366d-istiod-ca-cert\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536303 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536116    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:20:18.536303 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536134    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-envoy\") pod \"e58fe251-fa84-41b4-8b9b-97dd865d366d\" (UID: \"e58fe251-fa84-41b4-8b9b-97dd865d366d\") "
Sep 06 01:20:18.536303 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536231    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:20:18.536515 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536378    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:20:18.536515 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536455    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.536515 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536476    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-credential-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.536515 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536490    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-data\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.536917 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536466    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e58fe251-fa84-41b4-8b9b-97dd865d366d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 01:20:18.536917 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.536536    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:20:18.538254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.538227    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 01:20:18.538254 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.538238    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-kube-api-access-wcn58" (OuterVolumeSpecName: "kube-api-access-wcn58") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "kube-api-access-wcn58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:20:18.538383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.538348    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:20:18.538488 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.538474    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e58fe251-fa84-41b4-8b9b-97dd865d366d" (UID: "e58fe251-fa84-41b4-8b9b-97dd865d366d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:20:18.637041 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.637014    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-envoy\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.637041 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.637035    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcn58\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-kube-api-access-wcn58\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.637041 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.637045    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-podinfo\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.637218 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.637053    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fe251-fa84-41b4-8b9b-97dd865d366d-workload-socket\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.637218 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.637061    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fe251-fa84-41b4-8b9b-97dd865d366d-istio-token\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.637218 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.637068    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fe251-fa84-41b4-8b9b-97dd865d366d-istiod-ca-cert\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:20:18.818566 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.818536    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"]
Sep 06 01:20:18.824193 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:18.824172    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-470346-openshift-default-6554556bdb-jjb5r"]
Sep 06 01:20:20.202062 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:20.202013    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" path="/var/lib/kubelet/pods/e58fe251-fa84-41b4-8b9b-97dd865d366d/volumes"
Sep 06 01:20:38.324585 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:38.324559    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:20:38.334837 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:20:38.334819    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:21:35.929617 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:21:35.929583    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:21:37.033707 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:21:37.033670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:21:38.231831 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:21:38.231796    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:21:38.739308 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:21:38.739282    2575 scope.go:117] "RemoveContainer" containerID="59317a87a7abd2c9bced5d509b193f7a46fe23c050d11896c0bef5469185cc49"
Sep 06 01:21:38.747662 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:21:38.747645    2575 scope.go:117] "RemoveContainer" containerID="f0be12b9c3dd214709dafb5d51da16026d923644df99334b475428c8754c6d45"
Sep 06 01:22:11.592156 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:22:11.592115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:22:38.774958 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:22:38.774923    2575 scope.go:117] "RemoveContainer" containerID="4839dd6852c0988864b1ea4fff339f9b20f1b9e9f310f08577994e4b72f78a9f"
Sep 06 01:22:38.788833 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:22:38.788659    2575 scope.go:117] "RemoveContainer" containerID="14911a69f4f8d61fb855dabffb4f00acceb1cdbfbeaedeb6bacbe9b74f53faf1"
Sep 06 01:22:40.434687 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:22:40.434652    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:22:40.534608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:22:40.534577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:24:33.114228 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:33.114195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:24:38.823625 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:38.823595    2575 scope.go:117] "RemoveContainer" containerID="cd68192b10d7d3625665906c80e5c1c31fb99dcb4253cb6ce2a904c71f01b1c4"
Sep 06 01:24:56.675494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.675456    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"]
Sep 06 01:24:56.676063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.675912    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy"
Sep 06 01:24:56.676063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.675926    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy"
Sep 06 01:24:56.676063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.675950    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b" containerName="revoke-keys"
Sep 06 01:24:56.676063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.675959    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b" containerName="revoke-keys"
Sep 06 01:24:56.676063 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.676059    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy"
Sep 06 01:24:56.676253 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.676071    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="347e7f7f-834a-402c-9d96-295ac56f2d20" containerName="istio-proxy"
Sep 06 01:24:56.676253 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.676082    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="069a72f0-32a8-45b9-ad6a-e7a30cb6eb6b" containerName="revoke-keys"
Sep 06 01:24:56.676253 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.676093    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy"
Sep 06 01:24:56.679284 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.679268    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.681549 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.681529    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-9938c132-kserve-self-signed-certs\""
Sep 06 01:24:56.690916 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.690893    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"]
Sep 06 01:24:56.708672 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.708648    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:24:56.799018 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.798990    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tmp-dir\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.799169 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.799024    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stztf\" (UniqueName: \"kubernetes.io/projected/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-kube-api-access-stztf\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.799169 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.799069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-dshm\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.799169 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.799106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-home\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.799169 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.799122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-model-cache\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.799321 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.799202    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tls-certs\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900008 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.899976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-dshm\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900194 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-home\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900194 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900070    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-model-cache\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900194 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tls-certs\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900194 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tmp-dir\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900194 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900157    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stztf\" (UniqueName: \"kubernetes.io/projected/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-kube-api-access-stztf\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-model-cache\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900560 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900519    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-home\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.900601 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.900569    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tmp-dir\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.902492 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.902474    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-dshm\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.902773 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.902753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tls-certs\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.907695 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.907673    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stztf\" (UniqueName: \"kubernetes.io/projected/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-kube-api-access-stztf\") pod \"e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") " pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:56.996205 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:56.996176    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:24:57.275121 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:57.275098    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"]
Sep 06 01:24:57.277189 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:24:57.277159    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7c856ff_2c1c_4ab2_8a84_8d88c3510d6d.slice/crio-4fb7ccef88b867864233918ad6b3c030367cc2b2c0bd3ec9c7dd4085e2293cc9 WatchSource:0}: Error finding container 4fb7ccef88b867864233918ad6b3c030367cc2b2c0bd3ec9c7dd4085e2293cc9: Status 404 returned error can't find the container with id 4fb7ccef88b867864233918ad6b3c030367cc2b2c0bd3ec9c7dd4085e2293cc9
Sep 06 01:24:58.041566 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:58.041529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p" event={"ID":"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d","Type":"ContainerStarted","Data":"728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6"}
Sep 06 01:24:58.041566 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:58.041584    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p" event={"ID":"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d","Type":"ContainerStarted","Data":"4fb7ccef88b867864233918ad6b3c030367cc2b2c0bd3ec9c7dd4085e2293cc9"}
Sep 06 01:24:58.060642 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:24:58.060593    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p" podStartSLOduration=2.060579187 podStartE2EDuration="2.060579187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:24:58.058662106 +0000 UTC m=+2060.388020961" watchObservedRunningTime="2026-09-06 01:24:58.060579187 +0000 UTC m=+2060.389938023"
Sep 06 01:25:06.996892 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:06.996829    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:25:06.997348 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:06.997049    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:25:07.009956 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:07.009925    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:25:07.100790 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:07.100757    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:25:14.091114 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:14.091085    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:25:15.552387 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:15.552353    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"]
Sep 06 01:25:15.569345 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:15.569319    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:25:30.572586 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.572544    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p" podUID="b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" containerName="main" containerID="cri-o://728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6" gracePeriod=45
Sep 06 01:25:30.822122 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.822093    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:25:30.883393 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883314    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-dshm\") pod \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") "
Sep 06 01:25:30.883393 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883370    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tmp-dir\") pod \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") "
Sep 06 01:25:30.883615 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883396    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-model-cache\") pod \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") "
Sep 06 01:25:30.883615 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883431    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tls-certs\") pod \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") "
Sep 06 01:25:30.883615 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883452    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-home\") pod \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") "
Sep 06 01:25:30.883615 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883477    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stztf\" (UniqueName: \"kubernetes.io/projected/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-kube-api-access-stztf\") pod \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\" (UID: \"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d\") "
Sep 06 01:25:30.883810 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883693    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" (UID: "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:25:30.883810 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883701    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" (UID: "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:25:30.883810 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883720    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-home" (OuterVolumeSpecName: "home") pod "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" (UID: "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:25:30.883989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883833    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tmp-dir\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:25:30.883989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883878    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-model-cache\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:25:30.883989 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.883892    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-home\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:25:30.885696 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.885668    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-dshm" (OuterVolumeSpecName: "dshm") pod "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" (UID: "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 01:25:30.885802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.885723    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" (UID: "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 01:25:30.885802 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.885768    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-kube-api-access-stztf" (OuterVolumeSpecName: "kube-api-access-stztf") pod "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" (UID: "b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d"). InnerVolumeSpecName "kube-api-access-stztf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 01:25:30.984436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.984410    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-tls-certs\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:25:30.984436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.984432    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stztf\" (UniqueName: \"kubernetes.io/projected/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-kube-api-access-stztf\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:25:30.984436 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:30.984443    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d-dshm\") on node \"ip-10-0-129-5.ec2.internal\" DevicePath \"\""
Sep 06 01:25:31.221575 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.221538    2575 generic.go:356] "Generic (PLEG): container finished" podID="b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" containerID="728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6" exitCode=0
Sep 06 01:25:31.221752 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.221583    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"
Sep 06 01:25:31.221752 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.221624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p" event={"ID":"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d","Type":"ContainerDied","Data":"728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6"}
Sep 06 01:25:31.221752 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.221667    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p" event={"ID":"b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d","Type":"ContainerDied","Data":"4fb7ccef88b867864233918ad6b3c030367cc2b2c0bd3ec9c7dd4085e2293cc9"}
Sep 06 01:25:31.221752 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.221684    2575 scope.go:117] "RemoveContainer" containerID="728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6"
Sep 06 01:25:31.230954 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.230939    2575 scope.go:117] "RemoveContainer" containerID="728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6"
Sep 06 01:25:31.231236 ip-10-0-129-5 kubenswrapper[2575]: E0906 01:25:31.231218    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6\": container with ID starting with 728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6 not found: ID does not exist" containerID="728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6"
Sep 06 01:25:31.231294 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.231244    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6"} err="failed to get container status \"728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6\": rpc error: code = NotFound desc = could not find container \"728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6\": container with ID starting with 728992fa28bdacea75d676cdf2685749cac990a5f5384ff13630dd51ee8d0cd6 not found: ID does not exist"
Sep 06 01:25:31.241944 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.241925    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"]
Sep 06 01:25:31.246405 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:31.246384    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-9938c132-kserve-8c7f6fb75-4t25p"]
Sep 06 01:25:32.201478 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:32.201448    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" path="/var/lib/kubelet/pods/b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d/volumes"
Sep 06 01:25:38.363343 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:38.363319    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:25:38.372242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:38.372222    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qmkb4_239122a3-2aee-4587-b6ea-694cf69928cd/ovn-acl-logging/0.log"
Sep 06 01:25:38.844931 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:38.844897    2575 scope.go:117] "RemoveContainer" containerID="870eea5a80376a0c4ab8b7fef5a835375aed157f3be39190a6525153ab5ef714"
Sep 06 01:25:38.853468 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:38.853449    2575 scope.go:117] "RemoveContainer" containerID="e01ab4e1d1802dc0fc84b0fb9c4f651cda05ac903e80ca1772f22ea53cc683c8"
Sep 06 01:25:42.637203 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.637159    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"]
Sep 06 01:25:42.637657 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.637644    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy"
Sep 06 01:25:42.637728 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.637664    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58fe251-fa84-41b4-8b9b-97dd865d366d" containerName="istio-proxy"
Sep 06 01:25:42.637728 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.637694    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" containerName="main"
Sep 06 01:25:42.637728 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.637703    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" containerName="main"
Sep 06 01:25:42.637927 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.637806    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c856ff-2c1c-4ab2-8a84-8d88c3510d6d" containerName="main"
Sep 06 01:25:42.640826 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.640803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.644068 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.644045    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 01:25:42.644068 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.644061    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 01:25:42.644256 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.644070    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 06 01:25:42.644256 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.644054    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vb24r\""
Sep 06 01:25:42.644256 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.644053    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 06 01:25:42.649670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.649650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"]
Sep 06 01:25:42.785744 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.785709    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922f11c7-ed43-4681-8f6b-4bf19a997199-metrics-certs\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.785941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.785752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h52p8\" (UniqueName: \"kubernetes.io/projected/922f11c7-ed43-4681-8f6b-4bf19a997199-kube-api-access-h52p8\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.785941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.785777    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922f11c7-ed43-4681-8f6b-4bf19a997199-maas-api-tls\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.785941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.785847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922f11c7-ed43-4681-8f6b-4bf19a997199-openshift-service-ca\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.887061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.887030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922f11c7-ed43-4681-8f6b-4bf19a997199-openshift-service-ca\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.887198 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.887171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922f11c7-ed43-4681-8f6b-4bf19a997199-metrics-certs\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.887277 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.887209    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52p8\" (UniqueName: \"kubernetes.io/projected/922f11c7-ed43-4681-8f6b-4bf19a997199-kube-api-access-h52p8\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.887277 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.887234    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922f11c7-ed43-4681-8f6b-4bf19a997199-maas-api-tls\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.887714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.887693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922f11c7-ed43-4681-8f6b-4bf19a997199-openshift-service-ca\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.889784 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.889763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922f11c7-ed43-4681-8f6b-4bf19a997199-metrics-certs\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.889904 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.889838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922f11c7-ed43-4681-8f6b-4bf19a997199-maas-api-tls\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.895839 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.895821    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52p8\" (UniqueName: \"kubernetes.io/projected/922f11c7-ed43-4681-8f6b-4bf19a997199-kube-api-access-h52p8\") pod \"maas-api-6886844f88-ch2bs\" (UID: \"922f11c7-ed43-4681-8f6b-4bf19a997199\") " pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:42.952758 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:42.952729    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:43.231465 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:43.231433    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"]
Sep 06 01:25:43.233951 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:25:43.233920    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922f11c7_ed43_4681_8f6b_4bf19a997199.slice/crio-d449909e0945e75a6ba8296393a6d9b4f9ac5a1125ec7154a87f0d0c74649a61 WatchSource:0}: Error finding container d449909e0945e75a6ba8296393a6d9b4f9ac5a1125ec7154a87f0d0c74649a61: Status 404 returned error can't find the container with id d449909e0945e75a6ba8296393a6d9b4f9ac5a1125ec7154a87f0d0c74649a61
Sep 06 01:25:43.235240 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:43.235222    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 01:25:43.285407 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:43.285376    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs" event={"ID":"922f11c7-ed43-4681-8f6b-4bf19a997199","Type":"ContainerStarted","Data":"d449909e0945e75a6ba8296393a6d9b4f9ac5a1125ec7154a87f0d0c74649a61"}
Sep 06 01:25:45.300383 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:45.300347    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs" event={"ID":"922f11c7-ed43-4681-8f6b-4bf19a997199","Type":"ContainerStarted","Data":"0a44ac1d35097c1fb9d9e60e86bc18b5625bd7c85fdda4805190c95a8a08bb72"}
Sep 06 01:25:45.300779 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:45.300466    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:25:45.321492 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:45.321435    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs" podStartSLOduration=2.062314275 podStartE2EDuration="3.321421259s" totalImagesPullingTime="1.259106984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 01:25:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:25:45.320108959 +0000 UTC m=+2107.649467811" watchObservedRunningTime="2026-09-06 01:25:45.321421259 +0000 UTC m=+2107.650780095"
Sep 06 01:25:51.312157 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:25:51.312129    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6886844f88-ch2bs"
Sep 06 01:26:10.230768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:10.230734    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59567f46df-gg242_c5c69e02-23ab-4640-aaea-43146c344437/authorino/0.log"
Sep 06 01:26:16.338167 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:16.338141    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f8zs7_10b93760-2d2d-4c96-a1a4-df2830bf5ab9/server/0.log"
Sep 06 01:26:16.459929 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:16.459901    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2qs97_4155d25e-a9fd-40e0-b077-7558ef789d00/manager/0.log"
Sep 06 01:26:16.852435 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:16.852362    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79dff7ddb6-z8wrb_266694cc-ad09-4e8f-a7ad-bb3a0172564e/copy-manifests/0.log"
Sep 06 01:26:16.858608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:16.858585    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79dff7ddb6-z8wrb_266694cc-ad09-4e8f-a7ad-bb3a0172564e/manager/0.log"
Sep 06 01:26:17.214210 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:17.214184    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6886844f88-ch2bs_922f11c7-ed43-4681-8f6b-4bf19a997199/maas-api/0.log"
Sep 06 01:26:17.465456 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:17.465376    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29810955-rftq8_1b759535-2923-416b-a424-f22262735bde/cleanup/0.log"
Sep 06 01:26:18.409671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:18.409617    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59567f46df-gg242_c5c69e02-23ab-4640-aaea-43146c344437/authorino/0.log"
Sep 06 01:26:19.546670 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:19.546635    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc_a3175541-f613-4d89-af83-da8456e3bff1/istio-proxy/0.log"
Sep 06 01:26:19.914555 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:19.914452    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb67ffb98-z4cth_9946e0fe-8689-4a4e-866a-9f5007834772/kube-auth-proxy/0.log"
Sep 06 01:26:20.291730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:20.291698    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-rjs67_95231315-9b02-4114-9976-6af33cfc6af9/payload-processing/0.log"
Sep 06 01:26:21.136828 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:21.136797    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-sjd59_67dba8e3-064a-44cf-83e7-675b837d6610/main/0.log"
Sep 06 01:26:21.495190 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:21.495149    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-95tbs_09b0837d-b499-479b-82bc-b0e4dda0b55f/main/0.log"
Sep 06 01:26:27.089968 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:27.089936    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:26:27.188205 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:27.188177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log" limit=1024
Sep 06 01:26:30.058165 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:30.058084    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xsbhl_c5308096-acd0-49da-9d37-ef559fd0e363/global-pull-secret-syncer/0.log"
Sep 06 01:26:30.105357 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:30.105329    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2csv8_3ad2e909-80a7-454e-ae32-86257fc0c2db/konnectivity-agent/0.log"
Sep 06 01:26:30.200120 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:30.200095    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-5.ec2.internal_294d38c9919df9b4e3fbef92a3c28bbf/haproxy/0.log"
Sep 06 01:26:35.484730 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:35.484685    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59567f46df-gg242_c5c69e02-23ab-4640-aaea-43146c344437/authorino/0.log"
Sep 06 01:26:37.596827 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:37.596790    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26r2b_6c5965a0-d86f-4eaa-9d5f-0b1444b13047/node-exporter/0.log"
Sep 06 01:26:37.626768 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:37.626732    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26r2b_6c5965a0-d86f-4eaa-9d5f-0b1444b13047/kube-rbac-proxy/0.log"
Sep 06 01:26:37.664612 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:37.664586    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26r2b_6c5965a0-d86f-4eaa-9d5f-0b1444b13047/init-textfile/0.log"
Sep 06 01:26:38.221873 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.221822    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-drf4v_b238b9b4-70cc-411d-ad91-b0070261b17e/prometheus-operator-admission-webhook/0.log"
Sep 06 01:26:38.252242 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.252215    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-649b48b4cc-jpvk5_f13c41b5-bc0c-4db1-964c-adc0bcf56ff6/telemeter-client/0.log"
Sep 06 01:26:38.275200 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.275174    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-649b48b4cc-jpvk5_f13c41b5-bc0c-4db1-964c-adc0bcf56ff6/reload/0.log"
Sep 06 01:26:38.300216 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.300195    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-649b48b4cc-jpvk5_f13c41b5-bc0c-4db1-964c-adc0bcf56ff6/kube-rbac-proxy/0.log"
Sep 06 01:26:38.334646 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.334629    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cb79589c5-5v6fk_e2b097bd-a873-43f2-bbd5-82759cf0b6e0/thanos-query/0.log"
Sep 06 01:26:38.364589 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.364570    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cb79589c5-5v6fk_e2b097bd-a873-43f2-bbd5-82759cf0b6e0/kube-rbac-proxy-web/0.log"
Sep 06 01:26:38.387208 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.387184    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cb79589c5-5v6fk_e2b097bd-a873-43f2-bbd5-82759cf0b6e0/kube-rbac-proxy/0.log"
Sep 06 01:26:38.411941 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.411919    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cb79589c5-5v6fk_e2b097bd-a873-43f2-bbd5-82759cf0b6e0/prom-label-proxy/0.log"
Sep 06 01:26:38.435504 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.435481    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cb79589c5-5v6fk_e2b097bd-a873-43f2-bbd5-82759cf0b6e0/kube-rbac-proxy-rules/0.log"
Sep 06 01:26:38.457490 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.457471    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cb79589c5-5v6fk_e2b097bd-a873-43f2-bbd5-82759cf0b6e0/kube-rbac-proxy-metrics/0.log"
Sep 06 01:26:38.879093 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:38.879067    2575 scope.go:117] "RemoveContainer" containerID="70aa89164588b869e747f7af58fb0adae72d7fd22cb8741963f376bf30d5369d"
Sep 06 01:26:40.432205 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.432161    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"]
Sep 06 01:26:40.438461 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.438437    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.440714 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.440687    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4r8dp\"/\"default-dockercfg-9s6px\""
Sep 06 01:26:40.441207 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.441181    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4r8dp\"/\"openshift-service-ca.crt\""
Sep 06 01:26:40.441323 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.441254    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4r8dp\"/\"kube-root-ca.crt\""
Sep 06 01:26:40.443173 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.443150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"]
Sep 06 01:26:40.579251 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.579215    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sknhn\" (UniqueName: \"kubernetes.io/projected/39ccaea6-5d10-4139-9670-aa3d7fab8b07-kube-api-access-sknhn\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.579251 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.579251    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-podres\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.579508 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.579303    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-sys\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.579508 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.579352    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-lib-modules\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.579508 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.579380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-proc\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680569    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sknhn\" (UniqueName: \"kubernetes.io/projected/39ccaea6-5d10-4139-9670-aa3d7fab8b07-kube-api-access-sknhn\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680608 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680610    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-podres\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680654    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-sys\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-lib-modules\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-proc\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680799 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-sys\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680964 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-proc\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680964 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680826    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-podres\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.680964 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.680929    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ccaea6-5d10-4139-9670-aa3d7fab8b07-lib-modules\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.688717 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.688653    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sknhn\" (UniqueName: \"kubernetes.io/projected/39ccaea6-5d10-4139-9670-aa3d7fab8b07-kube-api-access-sknhn\") pod \"perf-node-gather-daemonset-5st94\" (UID: \"39ccaea6-5d10-4139-9670-aa3d7fab8b07\") " pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:40.749738 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:40.749716    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:41.021486 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.021460    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"]
Sep 06 01:26:41.023042 ip-10-0-129-5 kubenswrapper[2575]: W0906 01:26:41.023011    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod39ccaea6_5d10_4139_9670_aa3d7fab8b07.slice/crio-65fc17b2cd97d9a41ec3e12d366564f00691c1e5cb0c1476fd1f7d59fc77a8bb WatchSource:0}: Error finding container 65fc17b2cd97d9a41ec3e12d366564f00691c1e5cb0c1476fd1f7d59fc77a8bb: Status 404 returned error can't find the container with id 65fc17b2cd97d9a41ec3e12d366564f00691c1e5cb0c1476fd1f7d59fc77a8bb
Sep 06 01:26:41.068671 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.068651    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c9765c7c-xm4km_297eefe8-3829-4c27-b438-c2ff193162c9/console/0.log"
Sep 06 01:26:41.100988 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.100969    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-tnj9f_cea8e73c-c3ef-416e-ae8d-ef2110193236/download-server/0.log"
Sep 06 01:26:41.588045 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.588011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94" event={"ID":"39ccaea6-5d10-4139-9670-aa3d7fab8b07","Type":"ContainerStarted","Data":"17e59294d263abbee9f8bcc99a981f8a29a9dbd9df681a7a07a2f15bf06cb055"}
Sep 06 01:26:41.588496 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.588062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94" event={"ID":"39ccaea6-5d10-4139-9670-aa3d7fab8b07","Type":"ContainerStarted","Data":"65fc17b2cd97d9a41ec3e12d366564f00691c1e5cb0c1476fd1f7d59fc77a8bb"}
Sep 06 01:26:41.588496 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.588117    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:41.603403 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:41.603340    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94" podStartSLOduration=1.603327309 podStartE2EDuration="1.603327309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 01:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 01:26:41.60230392 +0000 UTC m=+2163.931662777" watchObservedRunningTime="2026-09-06 01:26:41.603327309 +0000 UTC m=+2163.932686144"
Sep 06 01:26:42.893480 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:42.893455    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gflfq_12e27939-fcf7-48ac-81f0-72a2bb057888/dns/0.log"
Sep 06 01:26:42.916494 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:42.916464    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gflfq_12e27939-fcf7-48ac-81f0-72a2bb057888/kube-rbac-proxy/0.log"
Sep 06 01:26:42.937960 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:42.937937    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pjw4s_4141a1cc-d62d-4397-9b44-097d6e0f3229/dns-node-resolver/0.log"
Sep 06 01:26:43.489700 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:43.489667    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-65d89dfff8-t7m85_1b5fdad7-cc5f-424e-af51-6a5b2fa77701/registry/0.log"
Sep 06 01:26:43.509517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:43.509496    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b9r8m_16c172de-2c01-4b51-a2ad-3e0b7b35ff3d/node-ca/0.log"
Sep 06 01:26:44.502558 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:44.502532    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc_a3175541-f613-4d89-af83-da8456e3bff1/istio-proxy/0.log"
Sep 06 01:26:45.080783 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:45.080755    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb67ffb98-z4cth_9946e0fe-8689-4a4e-866a-9f5007834772/kube-auth-proxy/0.log"
Sep 06 01:26:45.195990 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:45.195943    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-rjs67_95231315-9b02-4114-9976-6af33cfc6af9/payload-processing/0.log"
Sep 06 01:26:46.093157 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:46.093124    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-65gvx_10056f2c-e664-48ea-996d-a50e0dd2d41d/serve-healthcheck-canary/0.log"
Sep 06 01:26:46.782061 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:46.782006    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgg8h_e72cab94-f27f-4ce6-a5a8-95586bdc0955/kube-rbac-proxy/0.log"
Sep 06 01:26:46.802241 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:46.802215    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgg8h_e72cab94-f27f-4ce6-a5a8-95586bdc0955/exporter/0.log"
Sep 06 01:26:46.824754 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:46.824732    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgg8h_e72cab94-f27f-4ce6-a5a8-95586bdc0955/extractor/0.log"
Sep 06 01:26:47.603376 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:47.603348    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4r8dp/perf-node-gather-daemonset-5st94"
Sep 06 01:26:49.350517 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:49.350489    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f8zs7_10b93760-2d2d-4c96-a1a4-df2830bf5ab9/server/0.log"
Sep 06 01:26:49.427581 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:49.427555    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2qs97_4155d25e-a9fd-40e0-b077-7558ef789d00/manager/0.log"
Sep 06 01:26:49.619411 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:49.619379    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79dff7ddb6-z8wrb_266694cc-ad09-4e8f-a7ad-bb3a0172564e/manager/0.log"
Sep 06 01:26:49.643524 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:49.643501    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79dff7ddb6-z8wrb_266694cc-ad09-4e8f-a7ad-bb3a0172564e/copy-manifests/0.log"
Sep 06 01:26:50.710095 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:50.710063    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc_a3175541-f613-4d89-af83-da8456e3bff1/istio-proxy/0.log"
Sep 06 01:26:50.991664 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:50.991572    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c79fd5646-82vf2_a240ca61-e040-41d5-873d-ffbe0d8b682d/manager/0.log"
Sep 06 01:26:51.050018 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:51.049993    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-trx9n_a7cfb069-044d-4179-9a64-6f3466349795/openshift-lws-operator/0.log"
Sep 06 01:26:51.268050 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:51.267966    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb67ffb98-z4cth_9946e0fe-8689-4a4e-866a-9f5007834772/kube-auth-proxy/0.log"
Sep 06 01:26:51.358582 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:51.358555    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-rjs67_95231315-9b02-4114-9976-6af33cfc6af9/payload-processing/0.log"
Sep 06 01:26:52.868679 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:52.868653    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zgmhc_a3175541-f613-4d89-af83-da8456e3bff1/istio-proxy/0.log"
Sep 06 01:26:53.427981 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:53.427952    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cb67ffb98-z4cth_9946e0fe-8689-4a4e-866a-9f5007834772/kube-auth-proxy/0.log"
Sep 06 01:26:53.519525 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:53.519498    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-rjs67_95231315-9b02-4114-9976-6af33cfc6af9/payload-processing/0.log"
Sep 06 01:26:57.462086 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.462053    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/kube-multus-additional-cni-plugins/0.log"
Sep 06 01:26:57.485703 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.485679    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/egress-router-binary-copy/0.log"
Sep 06 01:26:57.505457 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.505437    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/cni-plugins/0.log"
Sep 06 01:26:57.526027 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.526006    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/bond-cni-plugin/0.log"
Sep 06 01:26:57.545996 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.545976    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/routeoverride-cni/0.log"
Sep 06 01:26:57.567093 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.567072    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/whereabouts-cni-bincopy/0.log"
Sep 06 01:26:57.589051 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.589026    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7txcq_6b7afb0c-24a3-4f9f-a295-e1d8c426e03c/whereabouts-cni/0.log"
Sep 06 01:26:57.950688 ip-10-0-129-5 kubenswrapper[2575]: I0906 01:26:57.950655    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c4mkc_551f2c61-9016-4ba6-94bf-ac19032e503e/kube-multus/0.log"