Aug 03 16:03:59.036124 ip-10-0-133-47 systemd[1]: Starting Kubernetes Kubelet...
Aug 03 16:03:59.472557 ip-10-0-133-47 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 16:03:59.472557 ip-10-0-133-47 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 03 16:03:59.472557 ip-10-0-133-47 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 16:03:59.472557 ip-10-0-133-47 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 03 16:03:59.472557 ip-10-0-133-47 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 03 16:03:59.474749 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.474665    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 03 16:03:59.482770 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482753    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:03:59.482770 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482769    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:03:59.482770 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482773    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482777    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482780    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482784    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482789    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482792    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482795    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482798    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482801    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482804    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482807    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482810    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482813    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482815    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482818    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482821    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482823    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482826    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482828    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482831    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:03:59.482893 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482834    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482836    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482839    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482841    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482843    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482846    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482849    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482851    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482854    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482856    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482859    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482861    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482876    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482879    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482883    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482887    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482890    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482893    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482896    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:03:59.483384 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482899    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482902    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482904    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482907    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482910    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482912    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482915    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482917    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482920    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482923    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482926    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482928    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482931    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482934    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482936    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482938    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482941    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482944    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482946    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482949    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:03:59.483864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482952    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482955    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482958    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482961    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482963    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482965    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482968    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482971    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482974    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482977    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482979    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482982    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482985    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482987    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482990    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482992    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482995    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.482997    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483000    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483003    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:03:59.484394 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483006    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483010    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483013    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483015    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483018    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483021    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483023    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483026    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483028    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483031    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483033    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483035    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483038    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483040    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483043    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483045    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483048    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483051    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483725    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:03:59.484904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483733    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483737    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483740    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483743    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483745    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483749    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483751    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483754    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483757    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483759    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483763    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483767    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483770    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483773    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483776    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483778    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483781    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483785    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483788    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:03:59.485458 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483791    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483794    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483797    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483799    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483802    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483805    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483807    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483810    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483812    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483815    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483817    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483819    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483822    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483825    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483828    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483830    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483833    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483835    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483838    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483841    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:03:59.485984 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483843    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483846    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483848    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483851    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483853    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483856    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483858    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483861    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483863    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483882    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483885    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483888    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483891    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483894    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483897    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483900    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483902    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483905    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483907    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483910    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:03:59.486501 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483912    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483915    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483918    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483920    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483922    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483927    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483930    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483933    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483936    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483938    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483941    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483943    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483946    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483949    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483952    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483955    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483958    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483960    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483963    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483965    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:03:59.487032 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483968    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483970    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483973    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483975    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483979    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483981    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483984    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483987    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483990    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483993    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483996    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.483998    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484001    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484003    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484006    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484008    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484011    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484014    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484017    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484411    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:03:59.487519 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484417    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484420    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484423    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484426    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484430    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484433    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484436    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484438    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484441    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484444    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484447    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484449    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484452    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484454    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484457    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484459    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484462    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484465    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484467    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:03:59.488034 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484470    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484472    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484475    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484478    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484481    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484484    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484486    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484489    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484493    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484497    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484500    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484503    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484507    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484510    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484513    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484516    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484528    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484531    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484541    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:03:59.488500 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484544    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484547    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484549    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484552    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484554    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484557    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484559    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484562    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484564    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484568    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484573    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484576    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484579    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484582    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484585    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484588    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484591    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484593    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484596    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:03:59.488983 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484599    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484601    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484604    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484606    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484609    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484612    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484615    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484617    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484620    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484622    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484625    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484627    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484630    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484633    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484635    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484638    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484640    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484643    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484645    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484648    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:03:59.489441 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484650    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484653    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484655    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484658    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484660    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484663    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484665    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484667    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484670    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484673    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484675    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484678    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484680    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484683    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484685    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484688    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484692    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484695    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484697    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484700    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.484702    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:03:59.489972 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484782    2560 flags.go:64] FLAG: --address="0.0.0.0"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484803    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484811    2560 flags.go:64] FLAG: --anonymous-auth="true"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484815    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484819    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484822    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484827    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484833    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484836    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484839    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484842    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484846    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484849    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484851    2560 flags.go:64] FLAG: --cgroup-root=""
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484854    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484857    2560 flags.go:64] FLAG: --client-ca-file=""
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484860    2560 flags.go:64] FLAG: --cloud-provider="external"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484863    2560 flags.go:64] FLAG: --cluster-dns="[]"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484880    2560 flags.go:64] FLAG: --cluster-domain=""
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484883    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484886    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484890    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484893    2560 flags.go:64] FLAG: --container-log-max-files="5"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484897    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 03 16:03:59.490475 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484900    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484903    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484906    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484911    2560 flags.go:64] FLAG: --contention-profiling="false"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484914    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484917    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484921    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484924    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484928    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484931    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484934    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484937    2560 flags.go:64] FLAG: --enable-load-reader="false"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484940    2560 flags.go:64] FLAG: --enable-server="true"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484943    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484948    2560 flags.go:64] FLAG: --event-burst="100"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484951    2560 flags.go:64] FLAG: --event-qps="50"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484953    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484957    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484959    2560 flags.go:64] FLAG: --eviction-hard=""
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484963    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484966    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484969    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484972    2560 flags.go:64] FLAG: --eviction-soft=""
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484974    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484977    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 03 16:03:59.491071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484980    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484983    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484986    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484989    2560 flags.go:64] FLAG: --fail-swap-on="true"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484992    2560 flags.go:64] FLAG: --feature-gates=""
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484996    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.484999    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485002    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485005    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485008    2560 flags.go:64] FLAG: --healthz-port="10248"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485012    2560 flags.go:64] FLAG: --help="false"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485015    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485018    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485021    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485024    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485027    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485031    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485034    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485036    2560 flags.go:64] FLAG: --image-service-endpoint=""
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485039    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485042    2560 flags.go:64] FLAG: --kube-api-burst="100"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485045    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485049    2560 flags.go:64] FLAG: --kube-api-qps="50"
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485052    2560 flags.go:64] FLAG: --kube-reserved=""
Aug 03 16:03:59.491727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485055    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485058    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485061    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485063    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485066    2560 flags.go:64] FLAG: --lock-file=""
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485069    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485071    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485074    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485079    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485082    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485085    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485088    2560 flags.go:64] FLAG: --logging-format="text"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485091    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485094    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485097    2560 flags.go:64] FLAG: --manifest-url=""
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485100    2560 flags.go:64] FLAG: --manifest-url-header=""
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485104    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485107    2560 flags.go:64] FLAG: --max-open-files="1000000"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485111    2560 flags.go:64] FLAG: --max-pods="110"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485114    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485117    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485120    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485123    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485126    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485129    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 03 16:03:59.492381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485132    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485139    2560 flags.go:64] FLAG: --node-status-max-images="50"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485142    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485145    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485149    2560 flags.go:64] FLAG: --pod-cidr=""
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485152    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485158    2560 flags.go:64] FLAG: --pod-manifest-path=""
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485161    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485164    2560 flags.go:64] FLAG: --pods-per-core="0"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485167    2560 flags.go:64] FLAG: --port="10250"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485170    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485173    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00112c27d9f9bab17"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485176    2560 flags.go:64] FLAG: --qos-reserved=""
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485179    2560 flags.go:64] FLAG: --read-only-port="10255"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485182    2560 flags.go:64] FLAG: --register-node="true"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485185    2560 flags.go:64] FLAG: --register-with-taints=""
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485188    2560 flags.go:64] FLAG: --registry-burst="10"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485191    2560 flags.go:64] FLAG: --registry-qps="5"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485196    2560 flags.go:64] FLAG: --reserved-cpus=""
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485199    2560 flags.go:64] FLAG: --reserved-memory=""
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485202    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485205    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485208    2560 flags.go:64] FLAG: --rotate-certificates="false"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485211    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485214    2560 flags.go:64] FLAG: --runonce="false"
Aug 03 16:03:59.493216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485217    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485220    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485223    2560 flags.go:64] FLAG: --seccomp-default="false"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485226    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485229    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485232    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485235    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485238    2560 flags.go:64] FLAG: --storage-driver-password="root"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485241    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485244    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485247    2560 flags.go:64] FLAG: --storage-driver-user="root"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485249    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485252    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485255    2560 flags.go:64] FLAG: --system-cgroups=""
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485258    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485264    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485266    2560 flags.go:64] FLAG: --tls-cert-file=""
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485269    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485274    2560 flags.go:64] FLAG: --tls-min-version=""
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485276    2560 flags.go:64] FLAG: --tls-private-key-file=""
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485279    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485282    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485285    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485288    2560 flags.go:64] FLAG: --v="2"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485292    2560 flags.go:64] FLAG: --version="false"
Aug 03 16:03:59.494131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485297    2560 flags.go:64] FLAG: --vmodule=""
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485301    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485304    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485415    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485419    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485422    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485424    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485427    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485430    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485432    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485435    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485438    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485441    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485443    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485446    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485448    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485451    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485453    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485456    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485458    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:03:59.494842 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485461    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485464    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485468    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485471    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485474    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485476    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485479    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485482    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485484    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485487    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485490    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485492    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485496    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485498    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485503    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485506    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485510    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485512    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485515    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485519    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:03:59.495367 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485522    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485525    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485527    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485530    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485532    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485535    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485538    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485541    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485543    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485546    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485548    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485551    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485553    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485556    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485558    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485561    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485564    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485566    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485569    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485572    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:03:59.495852 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485574    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485577    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485580    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485582    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485586    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485589    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485593    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485597    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485599    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485602    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485605    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485609    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485611    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485614    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485616    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485619    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485621    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485624    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485627    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:03:59.496410 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485629    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485632    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485634    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485636    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485639    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485642    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485646    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485649    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485651    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485655    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485657    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485660    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485663    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485666    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485669    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485672    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485674    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485678    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485681    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:03:59.496895 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485683    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485686    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485688    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.485691    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.485699    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.494322    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.494337    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.494357    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.494366    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494423    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494429    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494432    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494435    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494438    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494441    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:03:59.497371 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494444    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494447    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494450    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494452    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494455    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494457    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494460    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494463    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494465    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494468    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494470    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494473    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494475    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494480    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494483    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494486    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494489    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494492    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494494    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494497    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:03:59.497735 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494500    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494502    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494506    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494510    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494515    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494519    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494523    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494526    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494529    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494532    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494534    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494537    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494540    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494543    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494545    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494548    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494551    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494554    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494556    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:03:59.498305 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494559    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494561    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494564    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494567    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494569    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494572    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494574    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494577    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494580    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494583    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494585    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494588    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494591    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494594    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494596    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494599    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494602    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494605    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494608    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494610    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:03:59.498816 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494613    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494615    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494617    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494620    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494623    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494625    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494627    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494630    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494632    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494635    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494637    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494640    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494642    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494645    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494648    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494651    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494654    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494656    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494658    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494661    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:03:59.499321 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494663    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494666    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494669    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494671    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494674    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494676    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494678    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494681    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494684    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494686    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494689    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494692    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494694    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494697    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:03:59.499838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.494702    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494819    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494824    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494827    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494830    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494833    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494836    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494839    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494841    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494844    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494847    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494849    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494852    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494856    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494858    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494861    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494863    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494881    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494884    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494887    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 03 16:03:59.500212 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494890    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494894    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494898    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494901    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494904    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494907    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494910    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494913    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494916    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494919    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494922    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494925    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494927    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494930    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494933    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494935    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494938    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494940    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494943    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 03 16:03:59.500713 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494946    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494949    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494951    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494954    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494956    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494959    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494962    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494965    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494968    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494971    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494973    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494976    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494978    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494981    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494984    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494987    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494990    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494992    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494995    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 03 16:03:59.501189 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.494998    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495001    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495003    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495006    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495008    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495011    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495013    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495016    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495018    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495021    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495023    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495026    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495029    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495031    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495033    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495036    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495039    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495041    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495043    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495046    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 03 16:03:59.501639 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495049    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495051    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495054    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495056    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495059    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495061    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495064    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495066    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495069    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495072    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495075    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495077    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495080    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495083    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495085    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495088    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495090    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495094    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495097    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 03 16:03:59.502126 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495100    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495103    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:03:59.495106    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.495111    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.495215    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.498387    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.500041    2560 server.go:1019] "Starting client certificate rotation"
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.500138    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 03 16:03:59.502585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.500175    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 03 16:03:59.502924 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.502911    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 03 16:03:59.504969 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.504956    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 03 16:03:59.517631 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.517615    2560 log.go:25] "Validated CRI v1 runtime API"
Aug 03 16:03:59.520070 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.520056    2560 log.go:25] "Validated CRI v1 image API"
Aug 03 16:03:59.522752 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.522734    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 03 16:03:59.526668 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.526638    2560 fs.go:135] Filesystem UUIDs: map[2fddecc6-d27b-46c3-9d7a-92002c2bd965:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e2667986-3125-42f8-821c-485fee8e9ca9:/dev/nvme0n1p3]
Aug 03 16:03:59.526752 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.526668    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 03 16:03:59.526917 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.526902    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 03 16:03:59.532649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.532533    2560 manager.go:216] Machine: {Timestamp:2026-08-03 16:03:59.532059031 +0000 UTC m=+0.371003426 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103328 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec285ef3922da47881a3abfade346eae SystemUUID:ec285ef3-922d-a478-81a3-abfade346eae BootID:d4ebaa87-0186-40d4-a245-585cabb1eca3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f1:ea:5a:18:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f1:ea:5a:18:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:d7:75:73:4c:35 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 03 16:03:59.532649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.532648    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 03 16:03:59.532757 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.532731    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 03 16:03:59.535630 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.535603    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 03 16:03:59.536133 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.535632    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-47.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 03 16:03:59.536847 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.536836    2560 topology_manager.go:138] "Creating topology manager with none policy"
Aug 03 16:03:59.536887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.536849    2560 container_manager_linux.go:309] "Creating device plugin manager"
Aug 03 16:03:59.536887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.536857    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 03 16:03:59.537732 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.537721    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 03 16:03:59.539671 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.539661    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 03 16:03:59.545117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545104    2560 state_mem.go:36] "Initialized new in-memory state store"
Aug 03 16:03:59.545225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545214    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 03 16:03:59.545262 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545244    2560 kubelet.go:486] "Attempting to sync node with API server"
Aug 03 16:03:59.545262 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545253    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 03 16:03:59.545322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545267    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 03 16:03:59.545322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545276    2560 kubelet.go:398] "Adding apiserver pod source"
Aug 03 16:03:59.545322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.545285    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 03 16:03:59.546612 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.546587    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 03 16:03:59.546690 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.546620    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 03 16:03:59.549347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.549322    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 03 16:03:59.551235 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.551221    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 03 16:03:59.551295 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.551252    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 03 16:03:59.552006 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.551991    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 03 16:03:59.552049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552013    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 03 16:03:59.552049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552019    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 03 16:03:59.552049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552025    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 03 16:03:59.552049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552033    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 03 16:03:59.552049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552039    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 03 16:03:59.552183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552116    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 03 16:03:59.552183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552122    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 03 16:03:59.552183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552128    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 03 16:03:59.552183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552134    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 03 16:03:59.552183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552142    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 03 16:03:59.552183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.552150    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 03 16:03:59.553971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.553960    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 03 16:03:59.553971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.553971    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 03 16:03:59.555234 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.555217    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-47.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 03 16:03:59.555234 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.555222    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-47.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 03 16:03:59.555303 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.555273    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 03 16:03:59.557138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.557124    2560 server.go:1268] "Started kubelet"
Aug 03 16:03:59.557253 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.557208    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 03 16:03:59.557311 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.557299    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 03 16:03:59.557975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.557948    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 03 16:03:59.558038 ip-10-0-133-47 systemd[1]: Started Kubernetes Kubelet.
Aug 03 16:03:59.558786 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.558770    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 03 16:03:59.559701 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.559687    2560 server.go:310] "Adding debug handlers to kubelet server"
Aug 03 16:03:59.564075 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.563990    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 03 16:03:59.567322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.567305    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 03 16:03:59.567322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.567315    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 03 16:03:59.567681 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.567393    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:03:59.568062 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568046    2560 factory.go:55] Registering systemd factory
Aug 03 16:03:59.568179 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568169    2560 factory.go:223] Registration of the systemd container factory successfully
Aug 03 16:03:59.568288 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568057    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 03 16:03:59.568797 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568781    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 03 16:03:59.568905 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568801    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 03 16:03:59.568992 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568979    2560 reconstruct.go:97] "Volume reconstruction finished"
Aug 03 16:03:59.569044 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.568993    2560 reconciler.go:29] "Reconciler: start to sync state"
Aug 03 16:03:59.569345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.569330    2560 factory.go:153] Registering CRI-O factory
Aug 03 16:03:59.569345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.569347    2560 factory.go:223] Registration of the crio container factory successfully
Aug 03 16:03:59.569463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.569405    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 03 16:03:59.569463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.569428    2560 factory.go:103] Registering Raw factory
Aug 03 16:03:59.569463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.569442    2560 manager.go:1196] Started watching for new ooms in manager
Aug 03 16:03:59.570087 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.570064    2560 manager.go:349] Starting recovery of all containers
Aug 03 16:03:59.570625 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.570599    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-47.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 03 16:03:59.570720 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.570669    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 03 16:03:59.571849 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.570695    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-47.ec2.internal.18c8571e38452969  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-47.ec2.internal,UID:ip-10-0-133-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-47.ec2.internal,},FirstTimestamp:2026-08-03 16:03:59.557093737 +0000 UTC m=+0.396038132,LastTimestamp:2026-08-03 16:03:59.557093737 +0000 UTC m=+0.396038132,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-47.ec2.internal,}"
Aug 03 16:03:59.575619 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.575583    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7ksb"
Aug 03 16:03:59.581012 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.580987    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7ksb"
Aug 03 16:03:59.583206 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.583188    2560 manager.go:354] Recovery completed
Aug 03 16:03:59.587811 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.587797    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.589645 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.589628    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.589714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.589656    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.589714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.589666    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.590209 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590196    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 03 16:03:59.590209 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590207    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 03 16:03:59.590333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590214    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.590333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590222    2560 state_mem.go:36] "Initialized new in-memory state store"
Aug 03 16:03:59.590830 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590817    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.590882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590843    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.590882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.590853    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.591834 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.591766    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-47.ec2.internal.18c8571e3a35d456  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-47.ec2.internal,UID:ip-10-0-133-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-47.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-47.ec2.internal,},FirstTimestamp:2026-08-03 16:03:59.58964335 +0000 UTC m=+0.428587744,LastTimestamp:2026-08-03 16:03:59.58964335 +0000 UTC m=+0.428587744,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-47.ec2.internal,}"
Aug 03 16:03:59.592276 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.592263    2560 policy_none.go:49] "None policy: Start"
Aug 03 16:03:59.592332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.592279    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 03 16:03:59.592332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.592298    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 03 16:03:59.594298 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.594284    2560 policy_none.go:47] "Start"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626421    2560 manager.go:337] "Starting Device Plugin manager"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.626455    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626464    2560 server.go:87] "Starting device plugin registration server"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626688    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626699    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626786    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626855    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.626863    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.627367    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 03 16:03:59.642481 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.627420    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:03:59.689438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.689402    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 03 16:03:59.690626 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.690606    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 03 16:03:59.690737 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.690640    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 03 16:03:59.690737 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.690664    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 03 16:03:59.690737 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.690704    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 03 16:03:59.693312 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.693290    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:03:59.727637 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.727554    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.728882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.728855    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.728954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.728900    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.728954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.728911    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.728954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.728933    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.738928 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.738914    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.738983 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.738936    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-47.ec2.internal\": node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:03:59.761360 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.761341    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:03:59.791102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.791076    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"]
Aug 03 16:03:59.791199 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.791131    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.792596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.792583    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.792667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.792609    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.792667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.792618    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.794992 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.794980    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.795121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.795106    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.795182 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.795142    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.799391 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.799364    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.799465 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.799398    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.799465 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.799408    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.799465 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.799375    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.799560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.799471    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.799560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.799481    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.801691 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.801677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.801767 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.801699    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 03 16:03:59.802414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.802401    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientMemory"
Aug 03 16:03:59.802481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.802424    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasNoDiskPressure"
Aug 03 16:03:59.802481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.802437    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeHasSufficientPID"
Aug 03 16:03:59.822414 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.822393    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-47.ec2.internal\" not found" node="ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.825793 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.825777    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-47.ec2.internal\" not found" node="ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.862258 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.862236    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:03:59.870459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.870434    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f832807911ba67961295bcb6b0531fb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal\" (UID: \"f832807911ba67961295bcb6b0531fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.870554 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.870465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f832807911ba67961295bcb6b0531fb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal\" (UID: \"f832807911ba67961295bcb6b0531fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.870554 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.870484    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/002a780e41fc6d504e461c3d4d84078e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-47.ec2.internal\" (UID: \"002a780e41fc6d504e461c3d4d84078e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.962854 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:03:59.962650    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:03:59.971055 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.971029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f832807911ba67961295bcb6b0531fb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal\" (UID: \"f832807911ba67961295bcb6b0531fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.971109 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.971066    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f832807911ba67961295bcb6b0531fb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal\" (UID: \"f832807911ba67961295bcb6b0531fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.971109 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.971091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/002a780e41fc6d504e461c3d4d84078e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-47.ec2.internal\" (UID: \"002a780e41fc6d504e461c3d4d84078e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.971178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.971148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/002a780e41fc6d504e461c3d4d84078e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-47.ec2.internal\" (UID: \"002a780e41fc6d504e461c3d4d84078e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.971178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.971161    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f832807911ba67961295bcb6b0531fb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal\" (UID: \"f832807911ba67961295bcb6b0531fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:03:59.971240 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:03:59.971150    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f832807911ba67961295bcb6b0531fb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal\" (UID: \"f832807911ba67961295bcb6b0531fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:04:00.063524 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:00.063461    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:04:00.125002 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:00.124977    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:04:00.128148 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:00.128130    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"
Aug 03 16:04:00.164125 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:00.164099    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:04:00.264668 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:00.264631    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:04:00.365234 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:00.365166    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:04:01.755593 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:01.755565    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-47.ec2.internal\" not found"
Aug 03 16:04:01.772614 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.772596    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:01.788822 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.788798    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 03 16:04:01.788948 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.788930    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 03 16:04:01.789001 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.788957    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 03 16:04:01.835062 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.835035    2560 apiserver.go:52] "Watching apiserver"
Aug 03 16:04:01.854892 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.854862    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 03 16:04:01.855184 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.855164    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zkvxk","kube-system/konnectivity-agent-fl8dn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26","openshift-cluster-node-tuning-operator/tuned-8d74h","openshift-dns/node-resolver-mzdx8","openshift-image-registry/node-ca-xkv75","openshift-network-diagnostics/network-check-target-mdnkf","openshift-ovn-kubernetes/ovnkube-node-flzt6","openshift-multus/multus-additional-cni-plugins-xzwd5","openshift-multus/multus-fmb9l","openshift-multus/network-metrics-daemon-qmv94","openshift-network-operator/iptables-alerter-q99tr"]
Aug 03 16:04:01.857091 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.857072    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 03 16:04:01.857474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.857455    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"
Aug 03 16:04:01.857624 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.857607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.857720 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:01.857696    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:01.859818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.859796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.861941 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.861924    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.862473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.862454    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 03 16:04:01.862572 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.862519    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 03 16:04:01.862652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.862602    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gm6s2\""
Aug 03 16:04:01.864214 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864198    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.864214 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864207    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.864327 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf5d37db-5050-4930-b0a2-2869be2cc445-agent-certs\") pod \"konnectivity-agent-fl8dn\" (UID: \"bf5d37db-5050-4930-b0a2-2869be2cc445\") " pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.864327 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d37db-5050-4930-b0a2-2869be2cc445-konnectivity-ca\") pod \"konnectivity-agent-fl8dn\" (UID: \"bf5d37db-5050-4930-b0a2-2869be2cc445\") " pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.864327 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-kubelet-config\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.864472 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-dbus\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.864621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864603    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 03 16:04:01.864677 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864665    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.864779 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864749    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.864840 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.864827    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-glj2d\""
Aug 03 16:04:01.866648 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.866629    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.866742 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.866732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:01.866802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.866745    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.866802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.866760    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kzcvl\""
Aug 03 16:04:01.868408 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.868391    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 03 16:04:01.869445 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.869428    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:01.872028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.869780    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.872028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.869914    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.872028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.870028    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nll8c\""
Aug 03 16:04:01.872028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.870805    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 03 16:04:01.872028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.870904    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"
Aug 03 16:04:01.872028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.871846    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.872340 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.872185    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8skqn\""
Aug 03 16:04:01.872340 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.872305    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 03 16:04:01.872752 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.872734    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.873167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.873151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:01.873236 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:01.873211    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:01.875881 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.875866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.877945 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.877907    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-02 15:59:00 +0000 UTC" deadline="2028-02-01 15:39:14.241425157 +0000 UTC"
Aug 03 16:04:01.877945 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.877942    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13127h35m12.363484939s"
Aug 03 16:04:01.878411 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878388    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 03 16:04:01.878500 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878456    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 03 16:04:01.878500 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878466    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 03 16:04:01.878625 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878596    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.878679 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878640    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.878679 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878640    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 03 16:04:01.878773 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.878745    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.879483 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.879467    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ssntm\""
Aug 03 16:04:01.881211 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.881331 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881317    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.881400 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881335    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 03 16:04:01.881400 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881335    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 03 16:04:01.881492 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881451    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 03 16:04:01.881492 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881473    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kts2d\""
Aug 03 16:04:01.881641 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.881590    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.882217 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.882204    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 03 16:04:01.883818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.883803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:01.883912 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:01.883864    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:01.884701 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.884684    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 03 16:04:01.885118 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.885105    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v8l4s\""
Aug 03 16:04:01.886401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.886382    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal"]
Aug 03 16:04:01.886488 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.886412    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal"]
Aug 03 16:04:01.886488 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.886465    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:01.888897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.888860    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:01.889099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.889084    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x4ksf\""
Aug 03 16:04:01.889652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.889637    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:01.889708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.889694    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 03 16:04:01.959613 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.959593    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 03 16:04:01.965182 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-env-overrides\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.965256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965189    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-registration-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.965256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-modprobe-d\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.965256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-tuned\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.965256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08195d90-f93e-4d46-aae0-0eed9dad80b1-tmp\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.965395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-cnibin\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.965395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.965395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965338    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.965395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d37db-5050-4930-b0a2-2869be2cc445-konnectivity-ca\") pod \"konnectivity-agent-fl8dn\" (UID: \"bf5d37db-5050-4930-b0a2-2869be2cc445\") " pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.965510 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-var-lib-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.965510 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:01.965467    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:01.965588 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-cni-binary-copy\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.965619 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:01.965595    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:02.465568985 +0000 UTC m=+2.014855296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:01.965658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-kubelet\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.965693 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965652    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-run\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.965693 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965676    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-node-log\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.965751 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965702    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.965751 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965722    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-cni-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.965751 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-cnibin\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.965874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965767    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-netns\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.965874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-daemon-config\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.965874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2rm\" (UniqueName: \"kubernetes.io/projected/0ecec175-b462-4ce4-a083-b2192c1b83ab-kube-api-access-2q2rm\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:01.965874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf5d37db-5050-4930-b0a2-2869be2cc445-agent-certs\") pod \"konnectivity-agent-fl8dn\" (UID: \"bf5d37db-5050-4930-b0a2-2869be2cc445\") " pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.965986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.965986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-sys-fs\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.965986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965912    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysconfig\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.965986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-systemd-units\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.965983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf5d37db-5050-4930-b0a2-2869be2cc445-konnectivity-ca\") pod \"konnectivity-agent-fl8dn\" (UID: \"bf5d37db-5050-4930-b0a2-2869be2cc445\") " pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966015    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-os-release\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhcq2\" (UniqueName: \"kubernetes.io/projected/797df70c-a53e-4169-948e-91c5897af720-kube-api-access-lhcq2\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ecec175-b462-4ce4-a083-b2192c1b83ab-serviceca\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966062    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-socket-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966076    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysctl-conf\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.966095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966095    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr2lx\" (UniqueName: \"kubernetes.io/projected/3749bbe7-77cb-4a21-a66e-e9067b80b42f-kube-api-access-jr2lx\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-dbus\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw4nm\" (UniqueName: \"kubernetes.io/projected/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-kube-api-access-rw4nm\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966164    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxzr\" (UniqueName: \"kubernetes.io/projected/3a6b7921-0937-491f-83a8-8c30cb01d650-kube-api-access-flxzr\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-kubelet\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966227    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-system-cni-dir\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-os-release\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-k8s-cni-cncf-io\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-dbus\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.966345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-multus-certs\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966365    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ecec175-b462-4ce4-a083-b2192c1b83ab-host\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966391    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cd7dc32-3737-4e85-8d2b-294378dd796c-iptables-alerter-script\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-systemd\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-log-socket\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-cni-bin\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-etc-kubernetes\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovnkube-script-lib\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966615    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-device-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966639    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snw76\" (UniqueName: \"kubernetes.io/projected/17975002-99dd-4015-a156-db842277c1ad-kube-api-access-snw76\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-etc-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-cni-bin\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-kubelet-config\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966739    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-lib-modules\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.966859 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-ovn\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966792    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-kubelet-config\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966808    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-conf-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-529xp\" (UniqueName: \"kubernetes.io/projected/1cd7dc32-3737-4e85-8d2b-294378dd796c-kube-api-access-529xp\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-sys\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966905    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17975002-99dd-4015-a156-db842277c1ad-tmp-dir\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cd7dc32-3737-4e85-8d2b-294378dd796c-host-slash\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-kubernetes\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966966    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-slash\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.966983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovnkube-config\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967002    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-system-cni-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967015    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-var-lib-kubelet\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-systemd\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967077    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-cni-netd\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.967449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967129    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-cni-multus\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967185    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967208    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-host\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q87zx\" (UniqueName: \"kubernetes.io/projected/08195d90-f93e-4d46-aae0-0eed9dad80b1-kube-api-access-q87zx\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-run-netns\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brzgz\" (UniqueName: \"kubernetes.io/projected/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-kube-api-access-brzgz\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-hostroot\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967326    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysctl-d\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17975002-99dd-4015-a156-db842277c1ad-hosts-file\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-socket-dir-parent\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:01.968000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.967411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovn-node-metrics-cert\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:01.969466 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.969445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf5d37db-5050-4930-b0a2-2869be2cc445-agent-certs\") pod \"konnectivity-agent-fl8dn\" (UID: \"bf5d37db-5050-4930-b0a2-2869be2cc445\") " pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:01.978677 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.978660    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7cqjv"
Aug 03 16:04:01.983846 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:01.983828    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7cqjv"
Aug 03 16:04:02.020187 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.020151    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf832807911ba67961295bcb6b0531fb2.slice/crio-11d7066dca5adef4ddba207a04508c5b3cfd5d844b34802d81abda16d13ae76e WatchSource:0}: Error finding container 11d7066dca5adef4ddba207a04508c5b3cfd5d844b34802d81abda16d13ae76e: Status 404 returned error can't find the container with id 11d7066dca5adef4ddba207a04508c5b3cfd5d844b34802d81abda16d13ae76e
Aug 03 16:04:02.027094 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.027077    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:04:02.033981 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.033953    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002a780e41fc6d504e461c3d4d84078e.slice/crio-340934733fec073cdb30adb595e7a318de60a2bf60e701ad67a20394a712fde2 WatchSource:0}: Error finding container 340934733fec073cdb30adb595e7a318de60a2bf60e701ad67a20394a712fde2: Status 404 returned error can't find the container with id 340934733fec073cdb30adb595e7a318de60a2bf60e701ad67a20394a712fde2
Aug 03 16:04:02.068343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068309    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snw76\" (UniqueName: \"kubernetes.io/projected/17975002-99dd-4015-a156-db842277c1ad-kube-api-access-snw76\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.068343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-etc-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068357    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-cni-bin\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-lib-modules\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068409    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-ovn\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068424    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-conf-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068430    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-etc-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-529xp\" (UniqueName: \"kubernetes.io/projected/1cd7dc32-3737-4e85-8d2b-294378dd796c-kube-api-access-529xp\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-sys\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-ovn\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-conf-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-cni-bin\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068540    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-sys\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17975002-99dd-4015-a156-db842277c1ad-tmp-dir\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068581    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cd7dc32-3737-4e85-8d2b-294378dd796c-host-slash\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.068599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-kubernetes\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-slash\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-lib-modules\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovnkube-config\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068667    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-system-cni-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cd7dc32-3737-4e85-8d2b-294378dd796c-host-slash\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068681    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-slash\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068687    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-var-lib-kubelet\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.068737    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-system-cni-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-kubernetes\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.069100    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.069169    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:04:02.569142847 +0000 UTC m=+2.118429141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:02.069237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069178    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17975002-99dd-4015-a156-db842277c1ad-tmp-dir\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-systemd\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-cni-netd\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-run-systemd\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069435    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069471    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-cni-multus\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069498    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-host\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87zx\" (UniqueName: \"kubernetes.io/projected/08195d90-f93e-4d46-aae0-0eed9dad80b1-kube-api-access-q87zx\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-run-netns\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brzgz\" (UniqueName: \"kubernetes.io/projected/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-kube-api-access-brzgz\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069669    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-hostroot\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069697    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysctl-d\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17975002-99dd-4015-a156-db842277c1ad-hosts-file\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovnkube-config\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.069821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-cni-multus\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-hostroot\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-run-netns\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-socket-dir-parent\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovn-node-metrics-cert\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-env-overrides\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-registration-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.069974    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-modprobe-d\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-tuned\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-var-lib-kubelet\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08195d90-f93e-4d46-aae0-0eed9dad80b1-tmp\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-cnibin\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-cni-netd\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-var-lib-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070188    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-cni-binary-copy\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.070590 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysctl-d\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070233    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-host\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-kubelet\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070276    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-run\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-node-log\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070340    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070365    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-cni-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-cnibin\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-netns\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070453    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-daemon-config\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2rm\" (UniqueName: \"kubernetes.io/projected/0ecec175-b462-4ce4-a083-b2192c1b83ab-kube-api-access-2q2rm\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070512    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-sys-fs\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysconfig\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-systemd-units\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-os-release\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070797    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.071347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070874    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-kubelet\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.070686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhcq2\" (UniqueName: \"kubernetes.io/projected/797df70c-a53e-4169-948e-91c5897af720-kube-api-access-lhcq2\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ecec175-b462-4ce4-a083-b2192c1b83ab-serviceca\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-socket-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysctl-conf\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr2lx\" (UniqueName: \"kubernetes.io/projected/3749bbe7-77cb-4a21-a66e-e9067b80b42f-kube-api-access-jr2lx\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw4nm\" (UniqueName: \"kubernetes.io/projected/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-kube-api-access-rw4nm\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flxzr\" (UniqueName: \"kubernetes.io/projected/3a6b7921-0937-491f-83a8-8c30cb01d650-kube-api-access-flxzr\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-kubelet\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-system-cni-dir\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-os-release\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071420    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-k8s-cni-cncf-io\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071450    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-multus-certs\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ecec175-b462-4ce4-a083-b2192c1b83ab-host\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cd7dc32-3737-4e85-8d2b-294378dd796c-iptables-alerter-script\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-systemd\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.072107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-log-socket\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071612    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071643    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-cni-bin\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071696    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-etc-kubernetes\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071724    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovnkube-script-lib\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-device-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-device-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-run\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.071963    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-node-log\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.072452    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-registration-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.072479    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-cnibin\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.072562    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-var-lib-openvswitch\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.072785    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17975002-99dd-4015-a156-db842277c1ad-hosts-file\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.072922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.072905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-env-overrides\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-modprobe-d\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-tuned\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.072388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073219    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-cnibin\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073245    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/797df70c-a53e-4169-948e-91c5897af720-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-socket-dir-parent\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073302    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-cni-dir\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08195d90-f93e-4d46-aae0-0eed9dad80b1-tmp\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073436    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ecec175-b462-4ce4-a083-b2192c1b83ab-host\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073439    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-systemd\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073468    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-k8s-cni-cncf-io\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073496    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-var-lib-cni-bin\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.073596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073570    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-cni-binary-copy\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-socket-dir\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073622    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-os-release\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-systemd-units\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073641    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-etc-kubernetes\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073719    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073772    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-log-socket\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysctl-conf\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-kubelet\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.074142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.073892    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-multus-certs\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.074607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08195d90-f93e-4d46-aae0-0eed9dad80b1-etc-sysconfig\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.074607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-os-release\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.074607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074271    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.074607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074305    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/797df70c-a53e-4169-948e-91c5897af720-system-cni-dir\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.074607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-host-run-netns\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.074607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074447    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovnkube-script-lib\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.074834 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074611    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3a6b7921-0937-491f-83a8-8c30cb01d650-sys-fs\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.074834 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-multus-daemon-config\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.074897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.074866    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ecec175-b462-4ce4-a083-b2192c1b83ab-serviceca\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.075387 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.075371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-ovn-node-metrics-cert\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.075442 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.075427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cd7dc32-3737-4e85-8d2b-294378dd796c-iptables-alerter-script\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.077509 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.077490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snw76\" (UniqueName: \"kubernetes.io/projected/17975002-99dd-4015-a156-db842277c1ad-kube-api-access-snw76\") pod \"node-resolver-mzdx8\" (UID: \"17975002-99dd-4015-a156-db842277c1ad\") " pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.077731 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.077710    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-529xp\" (UniqueName: \"kubernetes.io/projected/1cd7dc32-3737-4e85-8d2b-294378dd796c-kube-api-access-529xp\") pod \"iptables-alerter-q99tr\" (UID: \"1cd7dc32-3737-4e85-8d2b-294378dd796c\") " pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.078339 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.078321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87zx\" (UniqueName: \"kubernetes.io/projected/08195d90-f93e-4d46-aae0-0eed9dad80b1-kube-api-access-q87zx\") pod \"tuned-8d74h\" (UID: \"08195d90-f93e-4d46-aae0-0eed9dad80b1\") " pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.078487 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.078471    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brzgz\" (UniqueName: \"kubernetes.io/projected/6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4-kube-api-access-brzgz\") pod \"ovnkube-node-flzt6\" (UID: \"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.081624 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.081605    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:02.081684 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.081631    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:02.081684 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.081644    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:02.081749 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.081702    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:02.581689271 +0000 UTC m=+2.130975570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:02.083982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.083956    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:02.084603 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.084580    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw4nm\" (UniqueName: \"kubernetes.io/projected/e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8-kube-api-access-rw4nm\") pod \"multus-fmb9l\" (UID: \"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8\") " pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.084805 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.084788    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhcq2\" (UniqueName: \"kubernetes.io/projected/797df70c-a53e-4169-948e-91c5897af720-kube-api-access-lhcq2\") pod \"multus-additional-cni-plugins-xzwd5\" (UID: \"797df70c-a53e-4169-948e-91c5897af720\") " pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.084866 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.084802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxzr\" (UniqueName: \"kubernetes.io/projected/3a6b7921-0937-491f-83a8-8c30cb01d650-kube-api-access-flxzr\") pod \"aws-ebs-csi-driver-node-rpl26\" (UID: \"3a6b7921-0937-491f-83a8-8c30cb01d650\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.084988 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.084972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2rm\" (UniqueName: \"kubernetes.io/projected/0ecec175-b462-4ce4-a083-b2192c1b83ab-kube-api-access-2q2rm\") pod \"node-ca-xkv75\" (UID: \"0ecec175-b462-4ce4-a083-b2192c1b83ab\") " pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.085112 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.085097    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr2lx\" (UniqueName: \"kubernetes.io/projected/3749bbe7-77cb-4a21-a66e-e9067b80b42f-kube-api-access-jr2lx\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:02.190591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.190496    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:02.199865 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.199836    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5d37db_5050_4930_b0a2_2869be2cc445.slice/crio-359286dfe7506fcb9daa04ce711120b4673bfa5c275ffec6435918ca4cf342fe WatchSource:0}: Error finding container 359286dfe7506fcb9daa04ce711120b4673bfa5c275ffec6435918ca4cf342fe: Status 404 returned error can't find the container with id 359286dfe7506fcb9daa04ce711120b4673bfa5c275ffec6435918ca4cf342fe
Aug 03 16:04:02.209845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.209827    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26"
Aug 03 16:04:02.217846 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.217825    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6b7921_0937_491f_83a8_8c30cb01d650.slice/crio-21cce2f13efbf80de6a40e1e42c9632cfe2c1833f5dafc94148c41b63467dda2 WatchSource:0}: Error finding container 21cce2f13efbf80de6a40e1e42c9632cfe2c1833f5dafc94148c41b63467dda2: Status 404 returned error can't find the container with id 21cce2f13efbf80de6a40e1e42c9632cfe2c1833f5dafc94148c41b63467dda2
Aug 03 16:04:02.220883 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.220867    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8d74h"
Aug 03 16:04:02.226984 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.226969    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mzdx8"
Aug 03 16:04:02.228599 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.228578    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08195d90_f93e_4d46_aae0_0eed9dad80b1.slice/crio-8bc2ace2cc530709c538cc05ea10db0502b3ddb13f9cefc4f19b7c622fd12e94 WatchSource:0}: Error finding container 8bc2ace2cc530709c538cc05ea10db0502b3ddb13f9cefc4f19b7c622fd12e94: Status 404 returned error can't find the container with id 8bc2ace2cc530709c538cc05ea10db0502b3ddb13f9cefc4f19b7c622fd12e94
Aug 03 16:04:02.236087 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.236066    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17975002_99dd_4015_a156_db842277c1ad.slice/crio-c6f8c5b0839f8a6e6dc1d3f4d109b1c5222bd7ef4fc06107d2b361c8da858710 WatchSource:0}: Error finding container c6f8c5b0839f8a6e6dc1d3f4d109b1c5222bd7ef4fc06107d2b361c8da858710: Status 404 returned error can't find the container with id c6f8c5b0839f8a6e6dc1d3f4d109b1c5222bd7ef4fc06107d2b361c8da858710
Aug 03 16:04:02.247825 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.247806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xkv75"
Aug 03 16:04:02.254469 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.254453    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:02.257093 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.257072    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecec175_b462_4ce4_a083_b2192c1b83ab.slice/crio-38660cf8a447694e6db62bb2e93c9a2302d7e2887632f50a1adece4c0c3a6261 WatchSource:0}: Error finding container 38660cf8a447694e6db62bb2e93c9a2302d7e2887632f50a1adece4c0c3a6261: Status 404 returned error can't find the container with id 38660cf8a447694e6db62bb2e93c9a2302d7e2887632f50a1adece4c0c3a6261
Aug 03 16:04:02.262803 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.262783    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a915c4c_a6d2_4cbb_acbf_a51d65e8e3d4.slice/crio-d4dee125d53ddc428340a6352952e382a681aec1dbcd5511be4d726cc0b45bb8 WatchSource:0}: Error finding container d4dee125d53ddc428340a6352952e382a681aec1dbcd5511be4d726cc0b45bb8: Status 404 returned error can't find the container with id d4dee125d53ddc428340a6352952e382a681aec1dbcd5511be4d726cc0b45bb8
Aug 03 16:04:02.270261 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.270245    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzwd5"
Aug 03 16:04:02.278912 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.278887    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod797df70c_a53e_4169_948e_91c5897af720.slice/crio-646bb4cd3a365bb4f534ca798fc16e97934fbd496df585094d49ed681c060e3b WatchSource:0}: Error finding container 646bb4cd3a365bb4f534ca798fc16e97934fbd496df585094d49ed681c060e3b: Status 404 returned error can't find the container with id 646bb4cd3a365bb4f534ca798fc16e97934fbd496df585094d49ed681c060e3b
Aug 03 16:04:02.286767 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.286749    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fmb9l"
Aug 03 16:04:02.291653 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.291635    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q99tr"
Aug 03 16:04:02.295074 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.295057    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e6bb1e_108f_4cd9_92d4_43e4db3aceb8.slice/crio-f021463c10f0e68a496bd88742a1668d7c7c0aa6b9629cfcce64cfd046a9696f WatchSource:0}: Error finding container f021463c10f0e68a496bd88742a1668d7c7c0aa6b9629cfcce64cfd046a9696f: Status 404 returned error can't find the container with id f021463c10f0e68a496bd88742a1668d7c7c0aa6b9629cfcce64cfd046a9696f
Aug 03 16:04:02.300160 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:02.300143    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd7dc32_3737_4e85_8d2b_294378dd796c.slice/crio-8d742b8b97d506b42523483e734233b408b1c0d2040464c338ffb61a11d1934a WatchSource:0}: Error finding container 8d742b8b97d506b42523483e734233b408b1c0d2040464c338ffb61a11d1934a: Status 404 returned error can't find the container with id 8d742b8b97d506b42523483e734233b408b1c0d2040464c338ffb61a11d1934a
Aug 03 16:04:02.410945 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.410916    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:02.474202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.474129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:02.474321 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.474245    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:02.474321 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.474292    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:03.474279419 +0000 UTC m=+3.023565713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:02.575597 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.575468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:02.575776 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.575650    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:02.575776 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.575719    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:04:03.575698233 +0000 UTC m=+3.124984552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:02.676755 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.676718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:02.677002 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.676932    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:02.677002 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.676955    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:02.677002 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.676968    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:02.677158 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:02.677020    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:03.677002195 +0000 UTC m=+3.226288489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:02.779012 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.778937    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:02.987037 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.986949    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 15:59:01 +0000 UTC" deadline="2028-02-21 21:58:24.39617622 +0000 UTC"
Aug 03 16:04:02.987037 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.986989    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13613h54m21.409191563s"
Aug 03 16:04:02.989274 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.989218    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"d4dee125d53ddc428340a6352952e382a681aec1dbcd5511be4d726cc0b45bb8"}
Aug 03 16:04:02.993102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.993069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xkv75" event={"ID":"0ecec175-b462-4ce4-a083-b2192c1b83ab","Type":"ContainerStarted","Data":"38660cf8a447694e6db62bb2e93c9a2302d7e2887632f50a1adece4c0c3a6261"}
Aug 03 16:04:02.996156 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.996131    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mzdx8" event={"ID":"17975002-99dd-4015-a156-db842277c1ad","Type":"ContainerStarted","Data":"c6f8c5b0839f8a6e6dc1d3f4d109b1c5222bd7ef4fc06107d2b361c8da858710"}
Aug 03 16:04:02.998429 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:02.998404    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8d74h" event={"ID":"08195d90-f93e-4d46-aae0-0eed9dad80b1","Type":"ContainerStarted","Data":"8bc2ace2cc530709c538cc05ea10db0502b3ddb13f9cefc4f19b7c622fd12e94"}
Aug 03 16:04:03.000299 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.000275    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26" event={"ID":"3a6b7921-0937-491f-83a8-8c30cb01d650","Type":"ContainerStarted","Data":"21cce2f13efbf80de6a40e1e42c9632cfe2c1833f5dafc94148c41b63467dda2"}
Aug 03 16:04:03.001849 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.001826    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fl8dn" event={"ID":"bf5d37db-5050-4930-b0a2-2869be2cc445","Type":"ContainerStarted","Data":"359286dfe7506fcb9daa04ce711120b4673bfa5c275ffec6435918ca4cf342fe"}
Aug 03 16:04:03.003907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.003882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q99tr" event={"ID":"1cd7dc32-3737-4e85-8d2b-294378dd796c","Type":"ContainerStarted","Data":"8d742b8b97d506b42523483e734233b408b1c0d2040464c338ffb61a11d1934a"}
Aug 03 16:04:03.010093 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.010068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fmb9l" event={"ID":"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8","Type":"ContainerStarted","Data":"f021463c10f0e68a496bd88742a1668d7c7c0aa6b9629cfcce64cfd046a9696f"}
Aug 03 16:04:03.013925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.013900    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerStarted","Data":"646bb4cd3a365bb4f534ca798fc16e97934fbd496df585094d49ed681c060e3b"}
Aug 03 16:04:03.016438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.016412    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal" event={"ID":"002a780e41fc6d504e461c3d4d84078e","Type":"ContainerStarted","Data":"340934733fec073cdb30adb595e7a318de60a2bf60e701ad67a20394a712fde2"}
Aug 03 16:04:03.018001 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.017982    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal" event={"ID":"f832807911ba67961295bcb6b0531fb2","Type":"ContainerStarted","Data":"11d7066dca5adef4ddba207a04508c5b3cfd5d844b34802d81abda16d13ae76e"}
Aug 03 16:04:03.483471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.483341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:03.483679 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.483513    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:03.483679 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.483602    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:05.483582917 +0000 UTC m=+5.032869233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:03.584113 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.584078    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:03.584337 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.584243    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:03.584337 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.584307    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:04:05.584290354 +0000 UTC m=+5.133576655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:03.685098 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.685060    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:03.685276 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.685235    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:03.685276 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.685256    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:03.685276 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.685271    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:03.685434 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.685330    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:05.685309564 +0000 UTC m=+5.234595878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:03.981671 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.980770    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:03.981671 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.980919    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:03.981671 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.981303    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:03.981671 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.981401    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:03.981671 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.981468    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:03.981671 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:03.981551    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:03.987842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.987781    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 15:59:01 +0000 UTC" deadline="2028-02-21 19:00:34.311957326 +0000 UTC"
Aug 03 16:04:03.987842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:03.987814    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13610h56m30.324149942s"
Aug 03 16:04:04.975482 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:04.975448    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 03 16:04:05.503902 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:05.503868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:05.504358 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.504028    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:05.504358 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.504109    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:09.50408926 +0000 UTC m=+9.053375553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:05.605379 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:05.605051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:05.605379 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.605212    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:05.605379 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.605280    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:04:09.605262122 +0000 UTC m=+9.154548420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:05.705822 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:05.705783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:05.706005 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.705977    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:05.706005 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.705995    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:05.706108 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.706008    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:05.706108 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.706064    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:09.706045813 +0000 UTC m=+9.255332113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:05.980995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:05.980823    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:05.980995 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.980978    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:05.980995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:05.980838    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:05.981275 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.981078    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:05.981275 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:05.980827    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:05.981275 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:05.981148    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:07.981384 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:07.981350    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:07.981977 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:07.981350    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:07.981977 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:07.981349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:07.981977 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:07.981559    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:07.981977 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:07.981629    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:07.981977 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:07.981723    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:09.536416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:09.536376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:09.536866 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.536561    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:09.536866 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.536644    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:17.536622572 +0000 UTC m=+17.085908872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:09.637717 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:09.637677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:09.637905 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.637885    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:09.637966 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.637960    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:04:17.637941281 +0000 UTC m=+17.187227583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:09.738719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:09.738628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:09.738883 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.738847    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:09.738883 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.738874    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:09.738988 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.738888    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:09.738988 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.738961    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:17.73893819 +0000 UTC m=+17.288224502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:09.981830 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:09.981022    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:09.981830 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.981155    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:09.981830 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:09.981558    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:09.981830 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.981644    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:09.981830 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:09.981710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:09.981830 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:09.981789    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:11.981952 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:11.981451    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:11.981952 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:11.981483    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:11.981952 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:11.981457    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:11.981952 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:11.981596    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:11.981952 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:11.981718    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:11.981952 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:11.981786    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:13.981434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:13.981399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:13.981434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:13.981441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:13.981955 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:13.981501    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:13.981955 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:13.981514    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:13.981955 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:13.981652    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:13.981955 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:13.981747    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:15.980834 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:15.980803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:15.981312 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:15.980803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:15.981312 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:15.980926    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:15.981312 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:15.980999    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:15.981312 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:15.980812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:15.981312 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:15.981085    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:17.597933 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:17.597903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:17.598349 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.598024    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:17.598349 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.598086    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:33.598068055 +0000 UTC m=+33.147354365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:17.698956 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:17.698923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:17.699060 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.699043    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:17.699110 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.699093    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:04:33.699076572 +0000 UTC m=+33.248362867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:17.799618 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:17.799587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:17.799800 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.799783    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:17.799846 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.799807    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:17.799846 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.799816    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:17.799931 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.799869    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:33.799852776 +0000 UTC m=+33.349139094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:17.980753 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:17.980671    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:17.980908 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:17.980673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:17.980908 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:17.980673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:17.981218 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.981185    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:17.982039 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.981953    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:17.982302 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:17.982258    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:19.980591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:19.980560    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:19.980591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:19.980580    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:19.981120 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:19.980603    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:19.981120 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:19.980697    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:19.981120 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:19.980828    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:19.981120 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:19.980902    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:21.981077 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:21.980920    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:21.981439 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:21.981186    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:21.981439 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:21.980981    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:21.981439 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:21.980938    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:21.981439 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:21.981290    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:21.981439 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:21.981358    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:23.090899 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.090660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mzdx8" event={"ID":"17975002-99dd-4015-a156-db842277c1ad","Type":"ContainerStarted","Data":"99381b72e81cbe42f7998be596ce52e227693e2359c2819141604a9f3e28cfdf"}
Aug 03 16:04:23.092093 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.092066    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8d74h" event={"ID":"08195d90-f93e-4d46-aae0-0eed9dad80b1","Type":"ContainerStarted","Data":"dc0263901cb9b1329772811d48bf87471278562e8ef574d8239cfc06474c932d"}
Aug 03 16:04:23.093555 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.093514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26" event={"ID":"3a6b7921-0937-491f-83a8-8c30cb01d650","Type":"ContainerStarted","Data":"70e705c5e1d5c657e136ecce38fe0566620157cb67f8c5daafea62beaba5d403"}
Aug 03 16:04:23.094682 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.094661    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fl8dn" event={"ID":"bf5d37db-5050-4930-b0a2-2869be2cc445","Type":"ContainerStarted","Data":"fea521f421038ae40e33e1a5a00be6ea17830d37c4ef7967780a3bf28295ae9d"}
Aug 03 16:04:23.095971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.095952    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fmb9l" event={"ID":"e4e6bb1e-108f-4cd9-92d4-43e4db3aceb8","Type":"ContainerStarted","Data":"76444a2619b4556787b390cda771eb2d76169021ce4218ad11921e49222362ac"}
Aug 03 16:04:23.097340 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.097319    2560 generic.go:356] "Generic (PLEG): container finished" podID="797df70c-a53e-4169-948e-91c5897af720" containerID="31b4cd1add57bcef4c0a72597a5ab2f6549ca2b89fdfc5f4dae0aa9846bcc21c" exitCode=0
Aug 03 16:04:23.097433 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.097379    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerDied","Data":"31b4cd1add57bcef4c0a72597a5ab2f6549ca2b89fdfc5f4dae0aa9846bcc21c"}
Aug 03 16:04:23.098839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.098820    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal" event={"ID":"002a780e41fc6d504e461c3d4d84078e","Type":"ContainerStarted","Data":"0b40b8169f9cfcb3eeb8988ee6e46967d7bd8f7a3f565b25573a6faad5d9ac06"}
Aug 03 16:04:23.100211 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.100190    2560 generic.go:356] "Generic (PLEG): container finished" podID="f832807911ba67961295bcb6b0531fb2" containerID="7f9fb2657c0c19d50369ae6f3df4c16a9085080c65c3ceeb33ed07c68cd313bc" exitCode=0
Aug 03 16:04:23.100317 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.100247    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal" event={"ID":"f832807911ba67961295bcb6b0531fb2","Type":"ContainerDied","Data":"7f9fb2657c0c19d50369ae6f3df4c16a9085080c65c3ceeb33ed07c68cd313bc"}
Aug 03 16:04:23.103127 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.103106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"b71b9080c12a552c009f1cb06e6686eab3051c3138732cf7c71faa4a12ac35cd"}
Aug 03 16:04:23.103205 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.103176    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"0cda10a3010e80ce2f8c206b08651443d96ee9b903286edfe11da77ec1e17551"}
Aug 03 16:04:23.103276 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.103260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"79548050eab227b4c634bc24969a569c62f41c1a3e0a406bc6303cfd6037fbd6"}
Aug 03 16:04:23.103337 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.103317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"84fda1c347456e8e25809fb586811b7b455e1fef3a84148671e4d6e5234e7011"}
Aug 03 16:04:23.103390 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.103356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"a10cb0d49728a501974378c77b918c5da2802a8652b3b4de2e1d2ec9c56b65a6"}
Aug 03 16:04:23.103438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.103390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"d002749cecc6ed5f1cc03b5d7866768a9487e0f0fb0bac212fa28217ba4b221b"}
Aug 03 16:04:23.104416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.104393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xkv75" event={"ID":"0ecec175-b462-4ce4-a083-b2192c1b83ab","Type":"ContainerStarted","Data":"c29f5473c7d3be4a7c2e49e7a8fd52566ee259877472277ec325cb92e68a33f4"}
Aug 03 16:04:23.122068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.122032    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mzdx8" podStartSLOduration=2.440443989 podStartE2EDuration="22.122020116s" totalImagesPullingTime="19.681576127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:23.106881359 +0000 UTC m=+22.656167676" watchObservedRunningTime="2026-08-03 16:04:23.122020116 +0000 UTC m=+22.671306433"
Aug 03 16:04:23.135576 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.135540    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-47.ec2.internal" podStartSLOduration=22.135513932 podStartE2EDuration="22.135513932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:23.135006787 +0000 UTC m=+22.684293115" watchObservedRunningTime="2026-08-03 16:04:23.135513932 +0000 UTC m=+22.684800249"
Aug 03 16:04:23.165993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.165545    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fl8dn" podStartSLOduration=2.447783288 podStartE2EDuration="22.165512504s" totalImagesPullingTime="19.717729216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:23.150073658 +0000 UTC m=+22.699359975" watchObservedRunningTime="2026-08-03 16:04:23.165512504 +0000 UTC m=+22.714798820"
Aug 03 16:04:23.165993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.165693    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8d74h" podStartSLOduration=2.478902417 podStartE2EDuration="22.165684735s" totalImagesPullingTime="19.686782318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:23.165660905 +0000 UTC m=+22.714947203" watchObservedRunningTime="2026-08-03 16:04:23.165684735 +0000 UTC m=+22.714971051"
Aug 03 16:04:23.186567 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.186406    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xkv75" podStartSLOduration=7.501616431 podStartE2EDuration="22.186394777s" totalImagesPullingTime="14.684778346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:23.185981459 +0000 UTC m=+22.735267776" watchObservedRunningTime="2026-08-03 16:04:23.186394777 +0000 UTC m=+22.735681093"
Aug 03 16:04:23.250307 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.250266    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fmb9l" podStartSLOduration=2.616558672 podStartE2EDuration="22.250252667s" totalImagesPullingTime="19.633693995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:23.249821158 +0000 UTC m=+22.799107475" watchObservedRunningTime="2026-08-03 16:04:23.250252667 +0000 UTC m=+22.799538983"
Aug 03 16:04:23.980899 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.980867    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:23.981141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.980911    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:23.981141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:23.980974    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:23.981141 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:23.980988    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:23.981141 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:23.981092    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:23.981356 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:23.981163    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:24.110444 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.110409    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q99tr" event={"ID":"1cd7dc32-3737-4e85-8d2b-294378dd796c","Type":"ContainerStarted","Data":"36f481b0395f2d91459b28d79abe0642629fc098abb5485f350b2c7435018950"}
Aug 03 16:04:24.112634 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.112605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal" event={"ID":"f832807911ba67961295bcb6b0531fb2","Type":"ContainerStarted","Data":"f08b59389b7bdde7b11d660ca5be164158fd85e5c05671ec64cf8cbac27c50f5"}
Aug 03 16:04:24.127891 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.127845    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q99tr" podStartSLOduration=3.5275216289999998 podStartE2EDuration="23.127832707s" totalImagesPullingTime="19.600311078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:24.127373122 +0000 UTC m=+23.676659431" watchObservedRunningTime="2026-08-03 16:04:24.127832707 +0000 UTC m=+23.677119022"
Aug 03 16:04:24.141968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.141934    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-47.ec2.internal" podStartSLOduration=23.141921611 podStartE2EDuration="23.141921611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:24.141903879 +0000 UTC m=+23.691190194" watchObservedRunningTime="2026-08-03 16:04:24.141921611 +0000 UTC m=+23.691207929"
Aug 03 16:04:24.432789 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.432752    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 03 16:04:24.932449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.932259    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-03T16:04:24.432773436Z","UUID":"c4a40b4b-cf27-4ab9-9ab9-53de10ce5ca6","Handler":null,"Name":"","Endpoint":""}
Aug 03 16:04:24.934278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.934254    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 03 16:04:24.934397 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:24.934295    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 03 16:04:25.118645 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.118607    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"a183ea2b16fb42d643d2d846a7bbd466d67d347234245c4f22fe1877244e6a66"}
Aug 03 16:04:25.120542 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.120481    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26" event={"ID":"3a6b7921-0937-491f-83a8-8c30cb01d650","Type":"ContainerStarted","Data":"25310ae7b55f3d7774ac2c3e7a5542cccdc98a58e0ab7f69fe9c08a1d3951440"}
Aug 03 16:04:25.981507 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.981480    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:25.981588 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.981513    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:25.981588 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.981513    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:25.981702 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:25.981677    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:25.981807 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:25.981790    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:25.981875 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:25.981860    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:25.996285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.996261    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:25.997001 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:25.996982    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:26.125246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:26.125213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26" event={"ID":"3a6b7921-0937-491f-83a8-8c30cb01d650","Type":"ContainerStarted","Data":"42dfeca50d6616b44e3ef154ebb063822ce9cda8adb2ba8cdc2ab4de9a332273"}
Aug 03 16:04:26.143688 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:26.143631    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpl26" podStartSLOduration=2.256256933 podStartE2EDuration="25.143610891s" totalImagesPullingTime="22.887353958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:26.142897769 +0000 UTC m=+25.692184084" watchObservedRunningTime="2026-08-03 16:04:26.143610891 +0000 UTC m=+25.692897209"
Aug 03 16:04:27.980719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:27.980687    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:27.981140 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:27.980720    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:27.981140 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:27.980770    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:27.981140 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:27.980851    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:27.981140 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:27.980998    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:27.981140 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:27.981090    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:28.133872 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:28.133334    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" event={"ID":"6a915c4c-a6d2-4cbb-acbf-a51d65e8e3d4","Type":"ContainerStarted","Data":"f9cde83e8fe53109898f426256a900f1482def6b22841ce6e1b795c65ac92009"}
Aug 03 16:04:28.133872 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:28.133668    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:28.133872 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:28.133727    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:28.151124 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:28.150869    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:28.151124 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:28.150968    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:28.160171 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:28.160132    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6" podStartSLOduration=7.238634766 podStartE2EDuration="27.160119278s" totalImagesPullingTime="19.921484512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:28.159712277 +0000 UTC m=+27.708998592" watchObservedRunningTime="2026-08-03 16:04:28.160119278 +0000 UTC m=+27.709405593"
Aug 03 16:04:29.136872 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:29.136837    2560 generic.go:356] "Generic (PLEG): container finished" podID="797df70c-a53e-4169-948e-91c5897af720" containerID="968e470a437798e1962229a9d465a4d785a8f1e9f9b97839f668560889a69bd5" exitCode=0
Aug 03 16:04:29.137577 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:29.136920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerDied","Data":"968e470a437798e1962229a9d465a4d785a8f1e9f9b97839f668560889a69bd5"}
Aug 03 16:04:29.137577 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:29.136958    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 03 16:04:29.980709 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:29.980679    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:29.980869 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:29.980733    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:29.980869 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:29.980844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:29.980869 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:29.980843    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:29.981031 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:29.981018    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:29.981096 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:29.981083    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:30.044156 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.044134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qmv94"]
Aug 03 16:04:30.047754 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.047730    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mdnkf"]
Aug 03 16:04:30.050820 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.050799    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zkvxk"]
Aug 03 16:04:30.140841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.140673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:30.140908    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.140694    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.140694    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.140689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerStarted","Data":"3d6334f1a9837447b9b8cee85ae8d3da69fcdd4d7ba3480f024c73fecbd5d47e"}
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:30.140977    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.140743    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 03 16:04:30.141146 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:30.141045    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:30.461121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.461090    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:30.461278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.461254    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 03 16:04:30.461697 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:30.461681    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fl8dn"
Aug 03 16:04:31.145591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:31.145560    2560 generic.go:356] "Generic (PLEG): container finished" podID="797df70c-a53e-4169-948e-91c5897af720" containerID="3d6334f1a9837447b9b8cee85ae8d3da69fcdd4d7ba3480f024c73fecbd5d47e" exitCode=0
Aug 03 16:04:31.146037 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:31.145635    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerDied","Data":"3d6334f1a9837447b9b8cee85ae8d3da69fcdd4d7ba3480f024c73fecbd5d47e"}
Aug 03 16:04:31.228440 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:31.228409    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:04:31.981599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:31.981509    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:31.981599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:31.981523    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:31.981599 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:31.981545    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:31.981824 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:31.981625    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:31.981824 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:31.981700    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:31.981824 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:31.981772    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:33.151574 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.151517    2560 generic.go:356] "Generic (PLEG): container finished" podID="797df70c-a53e-4169-948e-91c5897af720" containerID="1721571df7d844f519f52550fb8fc77efff48663f4d921183ca2dcfe52aa6ce2" exitCode=0
Aug 03 16:04:33.151574 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.151560    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerDied","Data":"1721571df7d844f519f52550fb8fc77efff48663f4d921183ca2dcfe52aa6ce2"}
Aug 03 16:04:33.616600 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.616519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:33.616766 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.616685    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:33.616766 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.616757    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret podName:cd3d71ba-13b7-41e1-a0ab-095fd1f4509c nodeName:}" failed. No retries permitted until 2026-08-03 16:05:05.616738041 +0000 UTC m=+65.166024349 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret") pod "global-pull-secret-syncer-zkvxk" (UID: "cd3d71ba-13b7-41e1-a0ab-095fd1f4509c") : object "kube-system"/"original-pull-secret" not registered
Aug 03 16:04:33.717736 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.717704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:33.717926 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.717851    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:33.717926 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.717904    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs podName:3749bbe7-77cb-4a21-a66e-e9067b80b42f nodeName:}" failed. No retries permitted until 2026-08-03 16:05:05.71789003 +0000 UTC m=+65.267176327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs") pod "network-metrics-daemon-qmv94" (UID: "3749bbe7-77cb-4a21-a66e-e9067b80b42f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 03 16:04:33.818412 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.818378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:33.818604 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.818498    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 03 16:04:33.818604 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.818512    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 03 16:04:33.818604 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.818522    2560 projected.go:196] Error preparing data for projected volume kube-api-access-h9f9k for pod openshift-network-diagnostics/network-check-target-mdnkf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:33.818604 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.818583    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k podName:ba9f5fca-83a4-45ad-a17d-8e49f27205e6 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:05.81856913 +0000 UTC m=+65.367855424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9f9k" (UniqueName: "kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k") pod "network-check-target-mdnkf" (UID: "ba9f5fca-83a4-45ad-a17d-8e49f27205e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 03 16:04:33.981473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.981398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:33.981654 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.981398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:33.981654 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:33.981398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:33.981654 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.981605    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkvxk" podUID="cd3d71ba-13b7-41e1-a0ab-095fd1f4509c"
Aug 03 16:04:33.981654 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.981490    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mdnkf" podUID="ba9f5fca-83a4-45ad-a17d-8e49f27205e6"
Aug 03 16:04:33.981654 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:33.981651    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmv94" podUID="3749bbe7-77cb-4a21-a66e-e9067b80b42f"
Aug 03 16:04:34.733903 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.733871    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-47.ec2.internal" event="NodeReady"
Aug 03 16:04:34.734312 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.734018    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 03 16:04:34.767146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.767118    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77dc6b9865-ptnlr"]
Aug 03 16:04:34.776814 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.776781    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-4xfp9"]
Aug 03 16:04:34.777068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.776881    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.780688 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.780363    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xchfl\""
Aug 03 16:04:34.780688 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.780593    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 03 16:04:34.780688 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.780686    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 03 16:04:34.780956 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.780938    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 03 16:04:34.787266 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.787244    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 03 16:04:34.794976 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.794277    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"]
Aug 03 16:04:34.794976 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.794672    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:34.797395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.797118    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Aug 03 16:04:34.797395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.797266    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p8tlg\""
Aug 03 16:04:34.797666 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.797650    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.797950 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.797934    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Aug 03 16:04:34.799116 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.799098    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.805095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.805074    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Aug 03 16:04:34.808607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.808585    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"]
Aug 03 16:04:34.809873 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.809849    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:34.812752 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.812723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Aug 03 16:04:34.813124 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.813093    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.813230 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.813166    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-w5wz5\""
Aug 03 16:04:34.813317 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.813298    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Aug 03 16:04:34.813393 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.813301    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.823028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.822995    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"]
Aug 03 16:04:34.823316 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.823290    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:34.828995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.828972    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s5pl7\""
Aug 03 16:04:34.830572 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.830551    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Aug 03 16:04:34.830737 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.830714    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.830995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.830553    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.831248 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.830901    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Aug 03 16:04:34.843522 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.843498    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-rt655"]
Aug 03 16:04:34.843637 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.843622    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:34.846061 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.846039    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.846204 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.846186    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2jdbc\""
Aug 03 16:04:34.846296 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.846095    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.846296 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.846157    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Aug 03 16:04:34.854583 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.854565    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"]
Aug 03 16:04:34.854714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.854696    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.857045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.857023    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Aug 03 16:04:34.857417 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.857358    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Aug 03 16:04:34.857670 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.857655    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.858046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.858028    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-89s8g\""
Aug 03 16:04:34.858124 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.858036    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.863026 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.863003    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Aug 03 16:04:34.869284 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869265    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-4xfp9"]
Aug 03 16:04:34.869375 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869300    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77dc6b9865-ptnlr"]
Aug 03 16:04:34.869375 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869328    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"]
Aug 03 16:04:34.869375 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869346    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-rt655"]
Aug 03 16:04:34.869375 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869365    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"]
Aug 03 16:04:34.869536 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"]
Aug 03 16:04:34.869536 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869409    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cltww"]
Aug 03 16:04:34.869622 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.869542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"
Aug 03 16:04:34.872128 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.872109    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.872256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.872238    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.872344 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.872328    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sxtr8\""
Aug 03 16:04:34.879433 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.879405    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7646c97b6d-frnzn"]
Aug 03 16:04:34.879561 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.879543    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:34.881826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.881808    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vz785\""
Aug 03 16:04:34.881940 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.881921    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 03 16:04:34.882018 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.881928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 03 16:04:34.894092 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.894075    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"]
Aug 03 16:04:34.894198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.894182    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:34.897100 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.897080    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 03 16:04:34.898463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.898445    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 03 16:04:34.898463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.898458    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8jpsd\""
Aug 03 16:04:34.898719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.898705    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.898996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.898978    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 03 16:04:34.899086 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.899013    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 03 16:04:34.899310 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.899221    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.908045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.908026    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l7gnr"]
Aug 03 16:04:34.908166 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.908149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:34.910655 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.910622    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.910918 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.910902    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Aug 03 16:04:34.910985 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.910939    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.911045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.911003    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fgtfp\""
Aug 03 16:04:34.911045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.911029    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Aug 03 16:04:34.920259 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.920242    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"]
Aug 03 16:04:34.920388 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.920370    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:34.923091 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.923074    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.923190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.923164    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 03 16:04:34.923243 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.923210    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dkjck\""
Aug 03 16:04:34.923300 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.923248    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.930296 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.930392 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-bound-sa-token\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.930392 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a79723-b930-4937-9c84-ac2f2b437efb-trusted-ca-bundle\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.930484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-installation-pull-secrets\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.930484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a79723-b930-4937-9c84-ac2f2b437efb-serving-cert\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.930484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930461    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qtbn\" (UniqueName: \"kubernetes.io/projected/fcc1d13c-3471-483a-80b1-0b7f726d7da7-kube-api-access-4qtbn\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:34.930629 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-certificates\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.930629 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3181b3d5-81de-4ea5-984c-d83d965219cc-trusted-ca\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:34.930629 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:34.930948 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.930571    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-image-registry-private-configuration\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/12a79723-b930-4937-9c84-ac2f2b437efb-snapshots\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931328    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-ca-trust-extracted\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-trusted-ca\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3181b3d5-81de-4ea5-984c-d83d965219cc-config\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a79723-b930-4937-9c84-ac2f2b437efb-tmp\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72977dd4-4353-44c8-a984-7b5c0445c7d0-serving-cert\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7npcl\" (UniqueName: \"kubernetes.io/projected/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-kube-api-access-7npcl\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntcz9\" (UniqueName: \"kubernetes.io/projected/3181b3d5-81de-4ea5-984c-d83d965219cc-kube-api-access-ntcz9\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931599    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a79723-b930-4937-9c84-ac2f2b437efb-service-ca-bundle\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931628    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgsz6\" (UniqueName: \"kubernetes.io/projected/12a79723-b930-4937-9c84-ac2f2b437efb-kube-api-access-dgsz6\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdb7x\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-kube-api-access-bdb7x\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931688    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3181b3d5-81de-4ea5-984c-d83d965219cc-serving-cert\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:34.932551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931713    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72977dd4-4353-44c8-a984-7b5c0445c7d0-config\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:34.933316 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.931774    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs4jz\" (UniqueName: \"kubernetes.io/projected/72977dd4-4353-44c8-a984-7b5c0445c7d0-kube-api-access-bs4jz\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:34.935063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.935021    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"]
Aug 03 16:04:34.935167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.935143    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:34.937867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.937845    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Aug 03 16:04:34.938083 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.938065    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Aug 03 16:04:34.938297 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.938282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ztcth\""
Aug 03 16:04:34.947192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947176    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947210    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cltww"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947237    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947256    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l7gnr"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947275    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947293    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947310    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7646c97b6d-frnzn"]
Aug 03 16:04:34.947347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.947319    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"
Aug 03 16:04:34.951585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.951562    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 03 16:04:34.951711 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.951592    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4k7km\""
Aug 03 16:04:34.951895 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.951882    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 03 16:04:34.961826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:34.961797    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:35.032785 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.032758    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.032975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.032800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a79723-b930-4937-9c84-ac2f2b437efb-serving-cert\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.032975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.032826    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7002810c-9381-4adb-95cb-f90d71adb444-config-volume\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.032975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.032924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72977dd4-4353-44c8-a984-7b5c0445c7d0-serving-cert\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.032975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.032959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgsz6\" (UniqueName: \"kubernetes.io/projected/12a79723-b930-4937-9c84-ac2f2b437efb-kube-api-access-dgsz6\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.033178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.032986    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.033178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp49j\" (UniqueName: \"kubernetes.io/projected/b6b62395-cc85-4e98-95ce-6c2fcbe9cb23-kube-api-access-hp49j\") pod \"network-check-source-65bdbbff45-kmm5q\" (UID: \"b6b62395-cc85-4e98-95ce-6c2fcbe9cb23\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"
Aug 03 16:04:35.033178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:35.033178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-image-registry-private-configuration\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.033370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033199    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7002810c-9381-4adb-95cb-f90d71adb444-tmp-dir\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.033370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033233    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.033370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/12a79723-b930-4937-9c84-ac2f2b437efb-snapshots\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.033370 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.033343    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 03 16:04:35.033611 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.033416    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls podName:fcc1d13c-3471-483a-80b1-0b7f726d7da7 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.533394172 +0000 UTC m=+35.082680467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-hzzkq" (UID: "fcc1d13c-3471-483a-80b1-0b7f726d7da7") : secret "samples-operator-tls" not found
Aug 03 16:04:35.033611 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-trusted-ca\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.033611 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033491    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3181b3d5-81de-4ea5-984c-d83d965219cc-config\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.033611 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033543    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bgbp\" (UniqueName: \"kubernetes.io/projected/957464d8-2258-4cce-acf5-0d5fecccebd9-kube-api-access-7bgbp\") pod \"ingress-canary-l7gnr\" (UID: \"957464d8-2258-4cce-acf5-0d5fecccebd9\") " pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.033611 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033577    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7npcl\" (UniqueName: \"kubernetes.io/projected/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-kube-api-access-7npcl\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:35.033611 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72977dd4-4353-44c8-a984-7b5c0445c7d0-config\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3181b3d5-81de-4ea5-984c-d83d965219cc-serving-cert\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033656    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjlb9\" (UniqueName: \"kubernetes.io/projected/15beb5d1-c300-43b2-ba75-232cc1167840-kube-api-access-sjlb9\") pod \"volume-data-source-validator-5b8776669-5tx54\" (UID: \"15beb5d1-c300-43b2-ba75-232cc1167840\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95g2\" (UniqueName: \"kubernetes.io/projected/7002810c-9381-4adb-95cb-f90d71adb444-kube-api-access-j95g2\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgd26\" (UniqueName: \"kubernetes.io/projected/0c729d6b-549e-4975-87be-65b77ffd61e4-kube-api-access-cgd26\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033791    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a79723-b930-4937-9c84-ac2f2b437efb-trusted-ca-bundle\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-default-certificate\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033844    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/12a79723-b930-4937-9c84-ac2f2b437efb-snapshots\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.033898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.033856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-installation-pull-secrets\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.034048    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.034062    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77dc6b9865-ptnlr: secret "image-registry-tls" not found
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.034108    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls podName:adbc06ec-cfa9-46ef-abbe-f1b14ec97dff nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.534090975 +0000 UTC m=+35.083377286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls") pod "image-registry-77dc6b9865-ptnlr" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff") : secret "image-registry-tls" not found
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qtbn\" (UniqueName: \"kubernetes.io/projected/fcc1d13c-3471-483a-80b1-0b7f726d7da7-kube-api-access-4qtbn\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034257    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-certificates\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3181b3d5-81de-4ea5-984c-d83d965219cc-trusted-ca\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.034335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5zp2\" (UniqueName: \"kubernetes.io/projected/50306696-a225-4a3f-b23a-5277b7d4563e-kube-api-access-l5zp2\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.034710 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034363    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-trusted-ca\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.034710 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034498    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3181b3d5-81de-4ea5-984c-d83d965219cc-config\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.035246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.034995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-certificates\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.035246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035091    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a79723-b930-4937-9c84-ac2f2b437efb-trusted-ca-bundle\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.035246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035211    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a79723-b930-4937-9c84-ac2f2b437efb-service-ca-bundle\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.035246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72977dd4-4353-44c8-a984-7b5c0445c7d0-config\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.035519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035303    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:35.035519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-ca-trust-extracted\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.035519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a79723-b930-4937-9c84-ac2f2b437efb-tmp\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.035519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs4jz\" (UniqueName: \"kubernetes.io/projected/72977dd4-4353-44c8-a984-7b5c0445c7d0-kube-api-access-bs4jz\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.035519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:35.035790 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035667    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a79723-b930-4937-9c84-ac2f2b437efb-service-ca-bundle\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.035790 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035777    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-ca-trust-extracted\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.035884 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035833    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c729d6b-549e-4975-87be-65b77ffd61e4-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.035884 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:35.035989 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035891    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a79723-b930-4937-9c84-ac2f2b437efb-tmp\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.035989 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntcz9\" (UniqueName: \"kubernetes.io/projected/3181b3d5-81de-4ea5-984c-d83d965219cc-kube-api-access-ntcz9\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.035989 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.035937    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-stats-auth\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.035989 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.035963    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:35.036181 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.036012    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls podName:c1c80cef-5998-485d-aacb-7d72d6a9d1fc nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.535995309 +0000 UTC m=+35.085281605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-p4b6q" (UID: "c1c80cef-5998-485d-aacb-7d72d6a9d1fc") : secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:35.036181 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.036140    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdb7x\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-kube-api-access-bdb7x\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.036293 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.036187    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:35.036348 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.036291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c729d6b-549e-4975-87be-65b77ffd61e4-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.036407 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.036347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957464d8-2258-4cce-acf5-0d5fecccebd9-cert\") pod \"ingress-canary-l7gnr\" (UID: \"957464d8-2258-4cce-acf5-0d5fecccebd9\") " pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.036459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.036433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-bound-sa-token\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.036616 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.036593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:35.038161 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.038137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72977dd4-4353-44c8-a984-7b5c0445c7d0-serving-cert\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.038161 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.038159    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-image-registry-private-configuration\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.038306 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.038148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a79723-b930-4937-9c84-ac2f2b437efb-serving-cert\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.038943 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.038908    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3181b3d5-81de-4ea5-984c-d83d965219cc-serving-cert\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.039068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.039045    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-installation-pull-secrets\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.042402 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.042379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgsz6\" (UniqueName: \"kubernetes.io/projected/12a79723-b930-4937-9c84-ac2f2b437efb-kube-api-access-dgsz6\") pod \"insights-operator-77696685b-rt655\" (UID: \"12a79723-b930-4937-9c84-ac2f2b437efb\") " pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.043296 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.043269    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3181b3d5-81de-4ea5-984c-d83d965219cc-trusted-ca\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.045349 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.045036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7npcl\" (UniqueName: \"kubernetes.io/projected/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-kube-api-access-7npcl\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:35.045349 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.045142    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntcz9\" (UniqueName: \"kubernetes.io/projected/3181b3d5-81de-4ea5-984c-d83d965219cc-kube-api-access-ntcz9\") pod \"console-operator-5ccd6b445f-4xfp9\" (UID: \"3181b3d5-81de-4ea5-984c-d83d965219cc\") " pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.045542 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.045400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qtbn\" (UniqueName: \"kubernetes.io/projected/fcc1d13c-3471-483a-80b1-0b7f726d7da7-kube-api-access-4qtbn\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:35.046225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.045988    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdb7x\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-kube-api-access-bdb7x\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.046225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.046149    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-bound-sa-token\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.046477 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.046429    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:35.047727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.047707    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs4jz\" (UniqueName: \"kubernetes.io/projected/72977dd4-4353-44c8-a984-7b5c0445c7d0-kube-api-access-bs4jz\") pod \"service-ca-operator-784f97bcf6-7jjz5\" (UID: \"72977dd4-4353-44c8-a984-7b5c0445c7d0\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.107077 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.107008    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:35.125897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.125875    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"
Aug 03 16:04:35.137478 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137450    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.137650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7002810c-9381-4adb-95cb-f90d71adb444-config-volume\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.137650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.137650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp49j\" (UniqueName: \"kubernetes.io/projected/b6b62395-cc85-4e98-95ce-6c2fcbe9cb23-kube-api-access-hp49j\") pod \"network-check-source-65bdbbff45-kmm5q\" (UID: \"b6b62395-cc85-4e98-95ce-6c2fcbe9cb23\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"
Aug 03 16:04:35.137650 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.137579    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 16:04:35.137650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137652    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7002810c-9381-4adb-95cb-f90d71adb444-tmp-dir\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137675    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.137720    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.637689167 +0000 UTC m=+35.186975464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : secret "router-metrics-certs-default" not found
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137771    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bgbp\" (UniqueName: \"kubernetes.io/projected/957464d8-2258-4cce-acf5-0d5fecccebd9-kube-api-access-7bgbp\") pod \"ingress-canary-l7gnr\" (UID: \"957464d8-2258-4cce-acf5-0d5fecccebd9\") " pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.137787    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.637770721 +0000 UTC m=+35.187057033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : configmap references non-existent config key: service-ca.crt
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137822    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjlb9\" (UniqueName: \"kubernetes.io/projected/15beb5d1-c300-43b2-ba75-232cc1167840-kube-api-access-sjlb9\") pod \"volume-data-source-validator-5b8776669-5tx54\" (UID: \"15beb5d1-c300-43b2-ba75-232cc1167840\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j95g2\" (UniqueName: \"kubernetes.io/projected/7002810c-9381-4adb-95cb-f90d71adb444-kube-api-access-j95g2\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgd26\" (UniqueName: \"kubernetes.io/projected/0c729d6b-549e-4975-87be-65b77ffd61e4-kube-api-access-cgd26\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.137946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-default-certificate\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.137981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5zp2\" (UniqueName: \"kubernetes.io/projected/50306696-a225-4a3f-b23a-5277b7d4563e-kube-api-access-l5zp2\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.137632    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138032    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.138102    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls podName:7002810c-9381-4adb-95cb-f90d71adb444 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.638058818 +0000 UTC m=+35.187345112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls") pod "dns-default-cltww" (UID: "7002810c-9381-4adb-95cb-f90d71adb444") : secret "dns-default-metrics-tls" not found
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c729d6b-549e-4975-87be-65b77ffd61e4-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.138139    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-stats-auth\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.138188    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert podName:3d8c27a8-7bad-47a2-ba46-2ab62d90b198 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:35.638173282 +0000 UTC m=+35.187459579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-wnbm6" (UID: "3d8c27a8-7bad-47a2-ba46-2ab62d90b198") : secret "networking-console-plugin-cert" not found
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c729d6b-549e-4975-87be-65b77ffd61e4-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138247    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7002810c-9381-4adb-95cb-f90d71adb444-tmp-dir\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.138381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.138270    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957464d8-2258-4cce-acf5-0d5fecccebd9-cert\") pod \"ingress-canary-l7gnr\" (UID: \"957464d8-2258-4cce-acf5-0d5fecccebd9\") " pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.139142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.139120    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c729d6b-549e-4975-87be-65b77ffd61e4-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.139232 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.139211    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:35.140389 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.140369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7002810c-9381-4adb-95cb-f90d71adb444-config-volume\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.142049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.142022    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-default-certificate\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.142200 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.142154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957464d8-2258-4cce-acf5-0d5fecccebd9-cert\") pod \"ingress-canary-l7gnr\" (UID: \"957464d8-2258-4cce-acf5-0d5fecccebd9\") " pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.142468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.142444    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-stats-auth\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.142575 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.142450    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c729d6b-549e-4975-87be-65b77ffd61e4-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.149652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.149197    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp49j\" (UniqueName: \"kubernetes.io/projected/b6b62395-cc85-4e98-95ce-6c2fcbe9cb23-kube-api-access-hp49j\") pod \"network-check-source-65bdbbff45-kmm5q\" (UID: \"b6b62395-cc85-4e98-95ce-6c2fcbe9cb23\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"
Aug 03 16:04:35.149652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.149228    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95g2\" (UniqueName: \"kubernetes.io/projected/7002810c-9381-4adb-95cb-f90d71adb444-kube-api-access-j95g2\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.149652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.149383    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bgbp\" (UniqueName: \"kubernetes.io/projected/957464d8-2258-4cce-acf5-0d5fecccebd9-kube-api-access-7bgbp\") pod \"ingress-canary-l7gnr\" (UID: \"957464d8-2258-4cce-acf5-0d5fecccebd9\") " pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.149652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.149613    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgd26\" (UniqueName: \"kubernetes.io/projected/0c729d6b-549e-4975-87be-65b77ffd61e4-kube-api-access-cgd26\") pod \"kube-storage-version-migrator-operator-7878f4b88c-9qzsb\" (UID: \"0c729d6b-549e-4975-87be-65b77ffd61e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.150711 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.150685    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjlb9\" (UniqueName: \"kubernetes.io/projected/15beb5d1-c300-43b2-ba75-232cc1167840-kube-api-access-sjlb9\") pod \"volume-data-source-validator-5b8776669-5tx54\" (UID: \"15beb5d1-c300-43b2-ba75-232cc1167840\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"
Aug 03 16:04:35.151052 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.151031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5zp2\" (UniqueName: \"kubernetes.io/projected/50306696-a225-4a3f-b23a-5277b7d4563e-kube-api-access-l5zp2\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.165120 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.165090    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-rt655"
Aug 03 16:04:35.179436 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.179414    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"
Aug 03 16:04:35.220356 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.220043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"
Aug 03 16:04:35.230434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.230400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l7gnr"
Aug 03 16:04:35.256932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.256895    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"
Aug 03 16:04:35.319236 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.319150    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5"]
Aug 03 16:04:35.324297 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.324236    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-4xfp9"]
Aug 03 16:04:35.324961 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.324890    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72977dd4_4353_44c8_a984_7b5c0445c7d0.slice/crio-794d356811cdc9c4231e7f4607acb2fc62667edb6ec120ff23e556de6b73c067 WatchSource:0}: Error finding container 794d356811cdc9c4231e7f4607acb2fc62667edb6ec120ff23e556de6b73c067: Status 404 returned error can't find the container with id 794d356811cdc9c4231e7f4607acb2fc62667edb6ec120ff23e556de6b73c067
Aug 03 16:04:35.342974 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.342903    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3181b3d5_81de_4ea5_984c_d83d965219cc.slice/crio-4c75ea183898c7d2778ab201d249f0360423af2e39962865c0c6c3f95a4340fd WatchSource:0}: Error finding container 4c75ea183898c7d2778ab201d249f0360423af2e39962865c0c6c3f95a4340fd: Status 404 returned error can't find the container with id 4c75ea183898c7d2778ab201d249f0360423af2e39962865c0c6c3f95a4340fd
Aug 03 16:04:35.375393 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.373010    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-rt655"]
Aug 03 16:04:35.381998 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.379956    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a79723_b930_4937_9c84_ac2f2b437efb.slice/crio-40ec1c92b62d98ab90be9e472c925567d7fee6a9c7ab1266b2263271dbdba9ad WatchSource:0}: Error finding container 40ec1c92b62d98ab90be9e472c925567d7fee6a9c7ab1266b2263271dbdba9ad: Status 404 returned error can't find the container with id 40ec1c92b62d98ab90be9e472c925567d7fee6a9c7ab1266b2263271dbdba9ad
Aug 03 16:04:35.432489 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.427037    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54"]
Aug 03 16:04:35.432873 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.432509    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15beb5d1_c300_43b2_ba75_232cc1167840.slice/crio-ea79800b31d8094d9d6c580e4ddeef1493875b5a170e517bc00c836c1627c15f WatchSource:0}: Error finding container ea79800b31d8094d9d6c580e4ddeef1493875b5a170e517bc00c836c1627c15f: Status 404 returned error can't find the container with id ea79800b31d8094d9d6c580e4ddeef1493875b5a170e517bc00c836c1627c15f
Aug 03 16:04:35.455314 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.455281    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb"]
Aug 03 16:04:35.458904 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.458796    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c729d6b_549e_4975_87be_65b77ffd61e4.slice/crio-2b88f325f29948d352d9d56d8b0c144129f7d6744e9cf03e16f03fbd21252e5f WatchSource:0}: Error finding container 2b88f325f29948d352d9d56d8b0c144129f7d6744e9cf03e16f03fbd21252e5f: Status 404 returned error can't find the container with id 2b88f325f29948d352d9d56d8b0c144129f7d6744e9cf03e16f03fbd21252e5f
Aug 03 16:04:35.474651 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.474385    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l7gnr"]
Aug 03 16:04:35.477068 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.477038    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957464d8_2258_4cce_acf5_0d5fecccebd9.slice/crio-2106f8512e75bf11b80549ee7e7a6763791f088928d1d542b649a2774568a035 WatchSource:0}: Error finding container 2106f8512e75bf11b80549ee7e7a6763791f088928d1d542b649a2774568a035: Status 404 returned error can't find the container with id 2106f8512e75bf11b80549ee7e7a6763791f088928d1d542b649a2774568a035
Aug 03 16:04:35.494092 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.494067    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q"]
Aug 03 16:04:35.497221 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:35.497198    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b62395_cc85_4e98_95ce_6c2fcbe9cb23.slice/crio-883489cff1d457217d233985259f94952b16082e926e1db163673c39f2b8d48c WatchSource:0}: Error finding container 883489cff1d457217d233985259f94952b16082e926e1db163673c39f2b8d48c: Status 404 returned error can't find the container with id 883489cff1d457217d233985259f94952b16082e926e1db163673c39f2b8d48c
Aug 03 16:04:35.543026 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.542993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:35.543168 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543146    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 03 16:04:35.543234 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543206    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls podName:fcc1d13c-3471-483a-80b1-0b7f726d7da7 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.543190386 +0000 UTC m=+36.092476684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-hzzkq" (UID: "fcc1d13c-3471-483a-80b1-0b7f726d7da7") : secret "samples-operator-tls" not found
Aug 03 16:04:35.543305 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.543151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:35.543435 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543233    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 16:04:35.543435 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543342    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77dc6b9865-ptnlr: secret "image-registry-tls" not found
Aug 03 16:04:35.543435 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543391    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:35.543435 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.543320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:35.543435 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543397    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls podName:adbc06ec-cfa9-46ef-abbe-f1b14ec97dff nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.543380525 +0000 UTC m=+36.092666857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls") pod "image-registry-77dc6b9865-ptnlr" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff") : secret "image-registry-tls" not found
Aug 03 16:04:35.543730 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.543455    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls podName:c1c80cef-5998-485d-aacb-7d72d6a9d1fc nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.543441941 +0000 UTC m=+36.092728236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-p4b6q" (UID: "c1c80cef-5998-485d-aacb-7d72d6a9d1fc") : secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:35.644401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.644321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:35.644401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.644386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.644658 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644497    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 16:04:35.644658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.644506    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:35.644658 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644583    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.644562554 +0000 UTC m=+36.193848849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : configmap references non-existent config key: service-ca.crt
Aug 03 16:04:35.644658 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644606    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 03 16:04:35.644658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.644651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:35.644893 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644664    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert podName:3d8c27a8-7bad-47a2-ba46-2ab62d90b198 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.644648725 +0000 UTC m=+36.193935025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-wnbm6" (UID: "3d8c27a8-7bad-47a2-ba46-2ab62d90b198") : secret "networking-console-plugin-cert" not found
Aug 03 16:04:35.644893 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644722    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls podName:7002810c-9381-4adb-95cb-f90d71adb444 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.64471502 +0000 UTC m=+36.194001314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls") pod "dns-default-cltww" (UID: "7002810c-9381-4adb-95cb-f90d71adb444") : secret "dns-default-metrics-tls" not found
Aug 03 16:04:35.644893 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644720    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 16:04:35.644893 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:35.644753    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:36.644748173 +0000 UTC m=+36.194034467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : secret "router-metrics-certs-default" not found
Aug 03 16:04:35.981172 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.981094    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:04:35.981172 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.981098    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:04:35.981986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.981224    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:04:35.984165 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.984141    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 03 16:04:35.984493 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.984471    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzpf\""
Aug 03 16:04:35.984493 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.984482    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xrdkc\""
Aug 03 16:04:35.984673 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:35.984486    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 03 16:04:36.160433 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.160397    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l7gnr" event={"ID":"957464d8-2258-4cce-acf5-0d5fecccebd9","Type":"ContainerStarted","Data":"2106f8512e75bf11b80549ee7e7a6763791f088928d1d542b649a2774568a035"}
Aug 03 16:04:36.162122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.162091    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54" event={"ID":"15beb5d1-c300-43b2-ba75-232cc1167840","Type":"ContainerStarted","Data":"ea79800b31d8094d9d6c580e4ddeef1493875b5a170e517bc00c836c1627c15f"}
Aug 03 16:04:36.164210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.164135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q" event={"ID":"b6b62395-cc85-4e98-95ce-6c2fcbe9cb23","Type":"ContainerStarted","Data":"883489cff1d457217d233985259f94952b16082e926e1db163673c39f2b8d48c"}
Aug 03 16:04:36.167314 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.167255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb" event={"ID":"0c729d6b-549e-4975-87be-65b77ffd61e4","Type":"ContainerStarted","Data":"2b88f325f29948d352d9d56d8b0c144129f7d6744e9cf03e16f03fbd21252e5f"}
Aug 03 16:04:36.170021 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.169998    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" event={"ID":"3181b3d5-81de-4ea5-984c-d83d965219cc","Type":"ContainerStarted","Data":"4c75ea183898c7d2778ab201d249f0360423af2e39962865c0c6c3f95a4340fd"}
Aug 03 16:04:36.171753 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.171693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-rt655" event={"ID":"12a79723-b930-4937-9c84-ac2f2b437efb","Type":"ContainerStarted","Data":"40ec1c92b62d98ab90be9e472c925567d7fee6a9c7ab1266b2263271dbdba9ad"}
Aug 03 16:04:36.173221 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.173167    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5" event={"ID":"72977dd4-4353-44c8-a984-7b5c0445c7d0","Type":"ContainerStarted","Data":"794d356811cdc9c4231e7f4607acb2fc62667edb6ec120ff23e556de6b73c067"}
Aug 03 16:04:36.554325 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.554061    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:36.554643 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.554414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:36.554643 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.554474    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:36.554643 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.554637    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:36.554830 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.554697    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls podName:c1c80cef-5998-485d-aacb-7d72d6a9d1fc nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.554678877 +0000 UTC m=+38.103965174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-p4b6q" (UID: "c1c80cef-5998-485d-aacb-7d72d6a9d1fc") : secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:36.555294 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.555089    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 03 16:04:36.555294 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.555145    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls podName:fcc1d13c-3471-483a-80b1-0b7f726d7da7 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.555128958 +0000 UTC m=+38.104415256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-hzzkq" (UID: "fcc1d13c-3471-483a-80b1-0b7f726d7da7") : secret "samples-operator-tls" not found
Aug 03 16:04:36.555294 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.555215    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 16:04:36.555294 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.555226    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77dc6b9865-ptnlr: secret "image-registry-tls" not found
Aug 03 16:04:36.555294 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.555261    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls podName:adbc06ec-cfa9-46ef-abbe-f1b14ec97dff nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.555248832 +0000 UTC m=+38.104535128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls") pod "image-registry-77dc6b9865-ptnlr" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff") : secret "image-registry-tls" not found
Aug 03 16:04:36.655818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.655750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:36.655818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.655812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:36.656065 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.655855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:36.656065 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:36.656017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:36.656201 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656176    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 03 16:04:36.656257 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656243    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert podName:3d8c27a8-7bad-47a2-ba46-2ab62d90b198 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.656224584 +0000 UTC m=+38.205510887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-wnbm6" (UID: "3d8c27a8-7bad-47a2-ba46-2ab62d90b198") : secret "networking-console-plugin-cert" not found
Aug 03 16:04:36.656349 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656335    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 16:04:36.656405 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656374    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.656362454 +0000 UTC m=+38.205648766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : secret "router-metrics-certs-default" not found
Aug 03 16:04:36.656463 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656434    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 16:04:36.656509 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656465    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls podName:7002810c-9381-4adb-95cb-f90d71adb444 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.656454833 +0000 UTC m=+38.205741130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls") pod "dns-default-cltww" (UID: "7002810c-9381-4adb-95cb-f90d71adb444") : secret "dns-default-metrics-tls" not found
Aug 03 16:04:36.656604 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:36.656549    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.656538923 +0000 UTC m=+38.205825233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : configmap references non-existent config key: service-ca.crt
Aug 03 16:04:38.574435 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.574396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574586    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.574621    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574661    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls podName:c1c80cef-5998-485d-aacb-7d72d6a9d1fc nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.57464095 +0000 UTC m=+42.123927247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-p4b6q" (UID: "c1c80cef-5998-485d-aacb-7d72d6a9d1fc") : secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574728    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.574741    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574782    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls podName:fcc1d13c-3471-483a-80b1-0b7f726d7da7 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.574770458 +0000 UTC m=+42.124056752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-hzzkq" (UID: "fcc1d13c-3471-483a-80b1-0b7f726d7da7") : secret "samples-operator-tls" not found
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574874    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574885    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77dc6b9865-ptnlr: secret "image-registry-tls" not found
Aug 03 16:04:38.574907 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.574912    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls podName:adbc06ec-cfa9-46ef-abbe-f1b14ec97dff nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.574904958 +0000 UTC m=+42.124191252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls") pod "image-registry-77dc6b9865-ptnlr" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff") : secret "image-registry-tls" not found
Aug 03 16:04:38.675471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.675439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:38.675650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.675502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:38.675822 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.675787    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 16:04:38.675918 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.675868    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.675848289 +0000 UTC m=+42.225134588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : secret "router-metrics-certs-default" not found
Aug 03 16:04:38.676252 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.676116    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 03 16:04:38.676252 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.676127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:38.676252 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.676178    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert podName:3d8c27a8-7bad-47a2-ba46-2ab62d90b198 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.67616363 +0000 UTC m=+42.225449933 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-wnbm6" (UID: "3d8c27a8-7bad-47a2-ba46-2ab62d90b198") : secret "networking-console-plugin-cert" not found
Aug 03 16:04:38.676252 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.676188    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 16:04:38.676252 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.676222    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls podName:7002810c-9381-4adb-95cb-f90d71adb444 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.676212663 +0000 UTC m=+42.225498962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls") pod "dns-default-cltww" (UID: "7002810c-9381-4adb-95cb-f90d71adb444") : secret "dns-default-metrics-tls" not found
Aug 03 16:04:38.676252 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:38.676242    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:38.676726 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:38.676358    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:42.676346356 +0000 UTC m=+42.225632650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : configmap references non-existent config key: service-ca.crt
Aug 03 16:04:39.645716 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:39.645664    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:39.706631 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:39.706596    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:40.084676 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:40.084643    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:40.184269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:40.184230    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:40.556583 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:40.556545    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:40.623689 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:40.623657    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:42.611543 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.611499    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.611641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611650    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611715    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls podName:c1c80cef-5998-485d-aacb-7d72d6a9d1fc nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.611698007 +0000 UTC m=+50.160984327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-p4b6q" (UID: "c1c80cef-5998-485d-aacb-7d72d6a9d1fc") : secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.611748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611787    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611850    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls podName:fcc1d13c-3471-483a-80b1-0b7f726d7da7 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.611832713 +0000 UTC m=+50.161119007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-hzzkq" (UID: "fcc1d13c-3471-483a-80b1-0b7f726d7da7") : secret "samples-operator-tls" not found
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611865    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611878    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77dc6b9865-ptnlr: secret "image-registry-tls" not found
Aug 03 16:04:42.611956 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.611906    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls podName:adbc06ec-cfa9-46ef-abbe-f1b14ec97dff nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.611897237 +0000 UTC m=+50.161183530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls") pod "image-registry-77dc6b9865-ptnlr" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff") : secret "image-registry-tls" not found
Aug 03 16:04:42.712151 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.712114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:42.712346 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.712178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:42.712346 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.712209    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:42.712346 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:42.712247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:42.712346 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712282    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 03 16:04:42.712346 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712335    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 16:04:42.712514 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712360    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert podName:3d8c27a8-7bad-47a2-ba46-2ab62d90b198 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.712336065 +0000 UTC m=+50.261622365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-wnbm6" (UID: "3d8c27a8-7bad-47a2-ba46-2ab62d90b198") : secret "networking-console-plugin-cert" not found
Aug 03 16:04:42.712514 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712379    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.712369637 +0000 UTC m=+50.261655935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : configmap references non-existent config key: service-ca.crt
Aug 03 16:04:42.712514 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712393    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 16:04:42.712514 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712396    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.712386732 +0000 UTC m=+50.261673030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : secret "router-metrics-certs-default" not found
Aug 03 16:04:42.712514 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:42.712439    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls podName:7002810c-9381-4adb-95cb-f90d71adb444 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.712424662 +0000 UTC m=+50.261710956 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls") pod "dns-default-cltww" (UID: "7002810c-9381-4adb-95cb-f90d71adb444") : secret "dns-default-metrics-tls" not found
Aug 03 16:04:43.352265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.352228    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:43.391354 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.391323    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"]
Aug 03 16:04:43.429387 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.429349    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:43.430120 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.430084    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"]
Aug 03 16:04:43.430281 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.430231    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.433071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.433046    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 03 16:04:43.434310 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.434287    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 03 16:04:43.434420 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.434287    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hjww6\""
Aug 03 16:04:43.434420 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.434295    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 03 16:04:43.434420 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.434338    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 03 16:04:43.446177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.446146    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"]
Aug 03 16:04:43.446286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.446186    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"]
Aug 03 16:04:43.446286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.446210    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"]
Aug 03 16:04:43.446286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.446250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.449653 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.449638    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 03 16:04:43.462255 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.462227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"]
Aug 03 16:04:43.462369 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.462352    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.465116 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.465094    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 03 16:04:43.465210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.465170    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 03 16:04:43.465269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.465209    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 03 16:04:43.465269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.465232    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 03 16:04:43.500293 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.500263    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:43.519662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519635    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxzdw\" (UniqueName: \"kubernetes.io/projected/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-kube-api-access-kxzdw\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.519788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519680    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/73602750-0eb5-4e17-9b15-43d035f18386-additional-service-ca\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.519788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519722    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-ca\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.519788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-hub\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.519788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.519966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519803    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.519966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-tmp\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.520043 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.519997    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/522bc6d9-acbc-4605-b3a1-b3ad0671258f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb\" (UID: \"522bc6d9-acbc-4605-b3a1-b3ad0671258f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.520043 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.520029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8fm5\" (UniqueName: \"kubernetes.io/projected/522bc6d9-acbc-4605-b3a1-b3ad0671258f-kube-api-access-c8fm5\") pod \"managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb\" (UID: \"522bc6d9-acbc-4605-b3a1-b3ad0671258f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.520138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.520064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.520192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.520143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7dwq\" (UniqueName: \"kubernetes.io/projected/73602750-0eb5-4e17-9b15-43d035f18386-kube-api-access-z7dwq\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.621573 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621469    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/73602750-0eb5-4e17-9b15-43d035f18386-additional-service-ca\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.621573 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-ca\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.621573 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621569    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-hub\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621671    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-tmp\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/522bc6d9-acbc-4605-b3a1-b3ad0671258f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb\" (UID: \"522bc6d9-acbc-4605-b3a1-b3ad0671258f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621767    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8fm5\" (UniqueName: \"kubernetes.io/projected/522bc6d9-acbc-4605-b3a1-b3ad0671258f-kube-api-access-c8fm5\") pod \"managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb\" (UID: \"522bc6d9-acbc-4605-b3a1-b3ad0671258f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621808    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.622122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.621840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dwq\" (UniqueName: \"kubernetes.io/projected/73602750-0eb5-4e17-9b15-43d035f18386-kube-api-access-z7dwq\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.622463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.622330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/73602750-0eb5-4e17-9b15-43d035f18386-additional-service-ca\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.622521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.622507    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxzdw\" (UniqueName: \"kubernetes.io/projected/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-kube-api-access-kxzdw\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.622713 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.622688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-tmp\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.625991 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.625940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.626106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.626061    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.626190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.626111    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/522bc6d9-acbc-4605-b3a1-b3ad0671258f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb\" (UID: \"522bc6d9-acbc-4605-b3a1-b3ad0671258f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.626414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.626392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-ca\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.626740 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.626721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-hub\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.626831 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.626744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/73602750-0eb5-4e17-9b15-43d035f18386-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.631107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.631083    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8fm5\" (UniqueName: \"kubernetes.io/projected/522bc6d9-acbc-4605-b3a1-b3ad0671258f-kube-api-access-c8fm5\") pod \"managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb\" (UID: \"522bc6d9-acbc-4605-b3a1-b3ad0671258f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.631219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.631192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxzdw\" (UniqueName: \"kubernetes.io/projected/fdd8d678-bd20-4277-a67f-ad2a42fcf14c-kube-api-access-kxzdw\") pod \"klusterlet-addon-workmgr-6fc4bb974b-rlnrg\" (UID: \"fdd8d678-bd20-4277-a67f-ad2a42fcf14c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.631277 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.631213    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7dwq\" (UniqueName: \"kubernetes.io/projected/73602750-0eb5-4e17-9b15-43d035f18386-kube-api-access-z7dwq\") pod \"cluster-proxy-proxy-agent-759dd77cff-qjv5n\" (UID: \"73602750-0eb5-4e17-9b15-43d035f18386\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:43.753233 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.753198    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"
Aug 03 16:04:43.763384 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.763320    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:43.772513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:43.772489    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:04:44.385219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:44.385186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:44.548026 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:44.547987    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:45.231170 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:45.231144    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"]
Aug 03 16:04:45.246464 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:45.246443    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb"]
Aug 03 16:04:45.354040 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:45.353997    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd8d678_bd20_4277_a67f_ad2a42fcf14c.slice/crio-a1b01a98b7a169b9071a73f07fcb12a28a19f6ab56ff2f6073bed42aa7d6c443 WatchSource:0}: Error finding container a1b01a98b7a169b9071a73f07fcb12a28a19f6ab56ff2f6073bed42aa7d6c443: Status 404 returned error can't find the container with id a1b01a98b7a169b9071a73f07fcb12a28a19f6ab56ff2f6073bed42aa7d6c443
Aug 03 16:04:45.354976 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:45.354952    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod522bc6d9_acbc_4605_b3a1_b3ad0671258f.slice/crio-941ee670006373ff130de73b1a505eabce84754571b824beb11febae91868b62 WatchSource:0}: Error finding container 941ee670006373ff130de73b1a505eabce84754571b824beb11febae91868b62: Status 404 returned error can't find the container with id 941ee670006373ff130de73b1a505eabce84754571b824beb11febae91868b62
Aug 03 16:04:45.455228 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:45.455207    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"]
Aug 03 16:04:45.461303 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:45.461278    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73602750_0eb5_4e17_9b15_43d035f18386.slice/crio-0635bd96bd31a28ead66306232e7c27d0c3523997cb81bf0ca39784686f29515 WatchSource:0}: Error finding container 0635bd96bd31a28ead66306232e7c27d0c3523997cb81bf0ca39784686f29515: Status 404 returned error can't find the container with id 0635bd96bd31a28ead66306232e7c27d0c3523997cb81bf0ca39784686f29515
Aug 03 16:04:46.212067 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.211356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l7gnr" event={"ID":"957464d8-2258-4cce-acf5-0d5fecccebd9","Type":"ContainerStarted","Data":"669d155144195699be175476176c9610162df5d6a412005fe285db13d1025241"}
Aug 03 16:04:46.216884 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.216447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54" event={"ID":"15beb5d1-c300-43b2-ba75-232cc1167840","Type":"ContainerStarted","Data":"903af4be00254f3393ccc75ef52fa2092d2c2d272e85a4de6f7d709caa467d71"}
Aug 03 16:04:46.220636 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.220573    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb" event={"ID":"522bc6d9-acbc-4605-b3a1-b3ad0671258f","Type":"ContainerStarted","Data":"941ee670006373ff130de73b1a505eabce84754571b824beb11febae91868b62"}
Aug 03 16:04:46.223058 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.222710    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q" event={"ID":"b6b62395-cc85-4e98-95ce-6c2fcbe9cb23","Type":"ContainerStarted","Data":"09d01ac4ecad12562bd08c7de6de642b0c9bd581228f126aabd29d7ef09b613f"}
Aug 03 16:04:46.229732 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.229708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb" event={"ID":"0c729d6b-549e-4975-87be-65b77ffd61e4","Type":"ContainerStarted","Data":"5e11eea88231cc5cec7cf75be57369c6e170d330cd6b82179ad4bfcbb85a60d1"}
Aug 03 16:04:46.231452 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.231407    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l7gnr" podStartSLOduration=2.63272577 podStartE2EDuration="12.231389471s" totalImagesPullingTime="9.598663701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.22941709 +0000 UTC m=+45.778703407" watchObservedRunningTime="2026-08-03 16:04:46.231389471 +0000 UTC m=+45.780675767"
Aug 03 16:04:46.237106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.236291    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/0.log"
Aug 03 16:04:46.237106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.236326    2560 generic.go:356] "Generic (PLEG): container finished" podID="3181b3d5-81de-4ea5-984c-d83d965219cc" containerID="6f1f74a361a4969d3acbcd3125f309d19b4dc4af4d948ff7cf8444383d29247b" exitCode=255
Aug 03 16:04:46.237106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.236548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" event={"ID":"3181b3d5-81de-4ea5-984c-d83d965219cc","Type":"ContainerDied","Data":"6f1f74a361a4969d3acbcd3125f309d19b4dc4af4d948ff7cf8444383d29247b"}
Aug 03 16:04:46.237106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.236970    2560 scope.go:117] "RemoveContainer" containerID="6f1f74a361a4969d3acbcd3125f309d19b4dc4af4d948ff7cf8444383d29247b"
Aug 03 16:04:46.241193 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.240595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-rt655" event={"ID":"12a79723-b930-4937-9c84-ac2f2b437efb","Type":"ContainerStarted","Data":"7898f752971541ee2581f7ab1783c0b498b086f1e1719c068908fdda7aace035"}
Aug 03 16:04:46.247183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.246792    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5" event={"ID":"72977dd4-4353-44c8-a984-7b5c0445c7d0","Type":"ContainerStarted","Data":"2ae4a204c1332a919fdde27955633cf0a4d1d10871be0bcd29a9b30419e4101d"}
Aug 03 16:04:46.254880 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.254626    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" event={"ID":"73602750-0eb5-4e17-9b15-43d035f18386","Type":"ContainerStarted","Data":"0635bd96bd31a28ead66306232e7c27d0c3523997cb81bf0ca39784686f29515"}
Aug 03 16:04:46.256300 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.256239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg" event={"ID":"fdd8d678-bd20-4277-a67f-ad2a42fcf14c","Type":"ContainerStarted","Data":"a1b01a98b7a169b9071a73f07fcb12a28a19f6ab56ff2f6073bed42aa7d6c443"}
Aug 03 16:04:46.264332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.263968    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-kmm5q" podStartSLOduration=14.686632111 podStartE2EDuration="24.263951558s" totalImagesPullingTime="9.577319447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.246152062 +0000 UTC m=+45.795438379" watchObservedRunningTime="2026-08-03 16:04:46.263951558 +0000 UTC m=+45.813237876"
Aug 03 16:04:46.266620 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.266068    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-5tx54" podStartSLOduration=14.729524649 podStartE2EDuration="24.266060071s" totalImagesPullingTime="9.536535422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.264780874 +0000 UTC m=+45.814067190" watchObservedRunningTime="2026-08-03 16:04:46.266060071 +0000 UTC m=+45.815346367"
Aug 03 16:04:46.270036 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.269760    2560 generic.go:356] "Generic (PLEG): container finished" podID="797df70c-a53e-4169-948e-91c5897af720" containerID="8d2009205f95597dc9ae19e6544fe609c26e67dbf8e7576adc7dfba00697dd9f" exitCode=0
Aug 03 16:04:46.270036 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.269794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerDied","Data":"8d2009205f95597dc9ae19e6544fe609c26e67dbf8e7576adc7dfba00697dd9f"}
Aug 03 16:04:46.289782 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.288910    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-rt655" podStartSLOduration=14.272850919 podStartE2EDuration="24.288890192s" totalImagesPullingTime="10.016039273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.287551118 +0000 UTC m=+45.836837433" watchObservedRunningTime="2026-08-03 16:04:46.288890192 +0000 UTC m=+45.838176510"
Aug 03 16:04:46.306826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.306146    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb" podStartSLOduration=14.698115292 podStartE2EDuration="24.306128227s" totalImagesPullingTime="9.608012935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.304894818 +0000 UTC m=+45.854181135" watchObservedRunningTime="2026-08-03 16:04:46.306128227 +0000 UTC m=+45.855414544"
Aug 03 16:04:46.322582 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.322499    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5" podStartSLOduration=14.574136109 podStartE2EDuration="24.322481171s" totalImagesPullingTime="9.748345062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.322022752 +0000 UTC m=+45.871309070" watchObservedRunningTime="2026-08-03 16:04:46.322481171 +0000 UTC m=+45.871767489"
Aug 03 16:04:46.388187 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.388130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:46.454487 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.454320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:46.783340 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:46.782517    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.283478 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.283392    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:04:47.285450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.284186    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/0.log"
Aug 03 16:04:47.285450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.284225    2560 generic.go:356] "Generic (PLEG): container finished" podID="3181b3d5-81de-4ea5-984c-d83d965219cc" containerID="6af8dbb52b4105a46cb01bcf9ec503c2ed7db9ddb31b43932e071471a4d9d362" exitCode=255
Aug 03 16:04:47.285450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.284319    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" event={"ID":"3181b3d5-81de-4ea5-984c-d83d965219cc","Type":"ContainerDied","Data":"6af8dbb52b4105a46cb01bcf9ec503c2ed7db9ddb31b43932e071471a4d9d362"}
Aug 03 16:04:47.285450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.284370    2560 scope.go:117] "RemoveContainer" containerID="6f1f74a361a4969d3acbcd3125f309d19b4dc4af4d948ff7cf8444383d29247b"
Aug 03 16:04:47.285450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.284770    2560 scope.go:117] "RemoveContainer" containerID="6af8dbb52b4105a46cb01bcf9ec503c2ed7db9ddb31b43932e071471a4d9d362"
Aug 03 16:04:47.285450 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:47.284966    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-4xfp9_openshift-console-operator(3181b3d5-81de-4ea5-984c-d83d965219cc)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" podUID="3181b3d5-81de-4ea5-984c-d83d965219cc"
Aug 03 16:04:47.299126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.299099    2560 generic.go:356] "Generic (PLEG): container finished" podID="797df70c-a53e-4169-948e-91c5897af720" containerID="809ca0df3306fa48b247856636cb08614c46050dabaa97fea68a6c3583974031" exitCode=0
Aug 03 16:04:47.300116 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.300083    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerDied","Data":"809ca0df3306fa48b247856636cb08614c46050dabaa97fea68a6c3583974031"}
Aug 03 16:04:47.347557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.347501    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.442739 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.442636    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.565014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.564926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.570432 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.566843    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"]
Aug 03 16:04:47.586423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.585456    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"]
Aug 03 16:04:47.586423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.585594    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"
Aug 03 16:04:47.590521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.589614    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2j8d7\""
Aug 03 16:04:47.590521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.589918    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Aug 03 16:04:47.590521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.590110    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Aug 03 16:04:47.656936 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.656884    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.666888 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.666666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqcqj\" (UniqueName: \"kubernetes.io/projected/d258a910-019a-4e1e-8a08-7e63343e0eba-kube-api-access-fqcqj\") pod \"migrator-68bc745d87-ddx7k\" (UID: \"d258a910-019a-4e1e-8a08-7e63343e0eba\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"
Aug 03 16:04:47.758176 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.758131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.768128 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.767952    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqcqj\" (UniqueName: \"kubernetes.io/projected/d258a910-019a-4e1e-8a08-7e63343e0eba-kube-api-access-fqcqj\") pod \"migrator-68bc745d87-ddx7k\" (UID: \"d258a910-019a-4e1e-8a08-7e63343e0eba\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"
Aug 03 16:04:47.786324 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.778731    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqcqj\" (UniqueName: \"kubernetes.io/projected/d258a910-019a-4e1e-8a08-7e63343e0eba-kube-api-access-fqcqj\") pod \"migrator-68bc745d87-ddx7k\" (UID: \"d258a910-019a-4e1e-8a08-7e63343e0eba\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"
Aug 03 16:04:47.823430 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.823335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:47.902186 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:47.901935    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"
Aug 03 16:04:48.045343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.045265    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.140619 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.140455    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.308082 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.308051    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:04:48.308513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.308492    2560 scope.go:117] "RemoveContainer" containerID="6af8dbb52b4105a46cb01bcf9ec503c2ed7db9ddb31b43932e071471a4d9d362"
Aug 03 16:04:48.308802 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:48.308774    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-4xfp9_openshift-console-operator(3181b3d5-81de-4ea5-984c-d83d965219cc)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" podUID="3181b3d5-81de-4ea5-984c-d83d965219cc"
Aug 03 16:04:48.318330 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.318266    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" event={"ID":"797df70c-a53e-4169-948e-91c5897af720","Type":"ContainerStarted","Data":"c6a23f32aeb0bce4cec118658521d61c6b2b7a98a40054390baeccceb5f07f50"}
Aug 03 16:04:48.343924 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.343829    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.352198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.352168    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k"]
Aug 03 16:04:48.546449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.546414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.645398 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.645311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.743801 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.743741    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xzwd5" podStartSLOduration=8.856557214 podStartE2EDuration="47.7437169s" totalImagesPullingTime="38.887159686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:48.352051391 +0000 UTC m=+47.901337708" watchObservedRunningTime="2026-08-03 16:04:48.7437169 +0000 UTC m=+48.293003216"
Aug 03 16:04:48.746027 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.745996    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ld9nn"]
Aug 03 16:04:48.767821 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.767788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.801699 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.801672    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ld9nn"]
Aug 03 16:04:48.801838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.801792    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.805569 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.805153    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-grgrw\""
Aug 03 16:04:48.805569 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.805225    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 03 16:04:48.805569 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.805382    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 03 16:04:48.827417 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.827392    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:48.880578 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.880544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10aead5f-7f93-4830-8756-f30d0e554e5c-crio-socket\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.880724 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.880630    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10aead5f-7f93-4830-8756-f30d0e554e5c-data-volume\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.880724 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.880682    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4pjd\" (UniqueName: \"kubernetes.io/projected/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-api-access-x4pjd\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.880839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.880731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.880839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.880820    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.981811 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.981730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4pjd\" (UniqueName: \"kubernetes.io/projected/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-api-access-x4pjd\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.981811 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.981808    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.981965 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.981858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.982010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.981992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10aead5f-7f93-4830-8756-f30d0e554e5c-crio-socket\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.982061 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.982044    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10aead5f-7f93-4830-8756-f30d0e554e5c-data-volume\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.982427 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.982403    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10aead5f-7f93-4830-8756-f30d0e554e5c-data-volume\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.982864 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:48.982843    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:48.982930 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:48.982923    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:49.482902521 +0000 UTC m=+49.032188831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:48.983178 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:48.983161    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 16:04:48.983247 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:48.983211    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:49.483196403 +0000 UTC m=+49.032482704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : configmap "kube-rbac-proxy" not found
Aug 03 16:04:48.983314 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.983267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10aead5f-7f93-4830-8756-f30d0e554e5c-crio-socket\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:48.999555 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:48.996287    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4pjd\" (UniqueName: \"kubernetes.io/projected/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-api-access-x4pjd\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:49.325023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.324595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k" event={"ID":"d258a910-019a-4e1e-8a08-7e63343e0eba","Type":"ContainerStarted","Data":"4aa117b2e1c0a874f3e4875564e30442ede60dbb496cc72da585460736b1d55f"}
Aug 03 16:04:49.488269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.488231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:49.488441 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:49.488375    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 16:04:49.488489 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.488440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:49.488489 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:49.488452    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.488433941 +0000 UTC m=+50.037720235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : configmap "kube-rbac-proxy" not found
Aug 03 16:04:49.488623 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:49.488554    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:49.488623 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:49.488605    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:50.488590552 +0000 UTC m=+50.037876856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:49.642894 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.642817    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-bdhtx"]
Aug 03 16:04:49.680968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.680932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:49.697075 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.697041    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-bdhtx"]
Aug 03 16:04:49.697233 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.697160    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.699910 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.699884    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 03 16:04:49.701139 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.701116    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 03 16:04:49.701139 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.701136    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xcrct\""
Aug 03 16:04:49.701337 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.701321    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 03 16:04:49.701441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.701398    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 03 16:04:49.782344 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.782310    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:49.792572 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.792515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrsxv\" (UniqueName: \"kubernetes.io/projected/3fb43105-598e-4db8-a872-f358c152ebb5-kube-api-access-mrsxv\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.792695 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.792624    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3fb43105-598e-4db8-a872-f358c152ebb5-signing-cabundle\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.792747 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.792731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3fb43105-598e-4db8-a872-f358c152ebb5-signing-key\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.894056 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.893973    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrsxv\" (UniqueName: \"kubernetes.io/projected/3fb43105-598e-4db8-a872-f358c152ebb5-kube-api-access-mrsxv\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.894056 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.894027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3fb43105-598e-4db8-a872-f358c152ebb5-signing-cabundle\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.894278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.894088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3fb43105-598e-4db8-a872-f358c152ebb5-signing-key\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.895013 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.894982    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3fb43105-598e-4db8-a872-f358c152ebb5-signing-cabundle\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.896901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.896857    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3fb43105-598e-4db8-a872-f358c152ebb5-signing-key\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:49.903315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:49.903290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrsxv\" (UniqueName: \"kubernetes.io/projected/3fb43105-598e-4db8-a872-f358c152ebb5-kube-api-access-mrsxv\") pod \"service-ca-98c88fbb5-bdhtx\" (UID: \"3fb43105-598e-4db8-a872-f358c152ebb5\") " pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:50.010038 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.009982    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx"
Aug 03 16:04:50.499385 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.499345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:50.499822 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.499477    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 03 16:04:50.499822 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.499580    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:52.499559213 +0000 UTC m=+52.048845508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : configmap "kube-rbac-proxy" not found
Aug 03 16:04:50.499822 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.499601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:50.499822 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.499740    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:50.499822 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.499782    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:52.499769399 +0000 UTC m=+52.049055694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:50.613658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.613487    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:50.701677 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.701640    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:04:50.701845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.701758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:04:50.701845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.701815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:04:50.701845 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.701819    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 03 16:04:50.702018 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.701886    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 03 16:04:50.702018 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.701908    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77dc6b9865-ptnlr: secret "image-registry-tls" not found
Aug 03 16:04:50.702018 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.701892    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls podName:fcc1d13c-3471-483a-80b1-0b7f726d7da7 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.701871313 +0000 UTC m=+66.251157626 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-hzzkq" (UID: "fcc1d13c-3471-483a-80b1-0b7f726d7da7") : secret "samples-operator-tls" not found
Aug 03 16:04:50.702018 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.701946    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:50.702217 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.701981    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls podName:adbc06ec-cfa9-46ef-abbe-f1b14ec97dff nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.701961612 +0000 UTC m=+66.251247913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls") pod "image-registry-77dc6b9865-ptnlr" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff") : secret "image-registry-tls" not found
Aug 03 16:04:50.702217 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.702056    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls podName:c1c80cef-5998-485d-aacb-7d72d6a9d1fc nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.702039234 +0000 UTC m=+66.251325535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-p4b6q" (UID: "c1c80cef-5998-485d-aacb-7d72d6a9d1fc") : secret "cluster-monitoring-operator-tls" not found
Aug 03 16:04:50.802648 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.802614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:50.802838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.802673    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:04:50.802838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.802723    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:04:50.802961 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.802845    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.802831397 +0000 UTC m=+66.352117704 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : configmap references non-existent config key: service-ca.crt
Aug 03 16:04:50.802961 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.802847    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 03 16:04:50.802961 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.802847    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 03 16:04:50.802961 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.802904    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls podName:7002810c-9381-4adb-95cb-f90d71adb444 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.80289022 +0000 UTC m=+66.352176513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls") pod "dns-default-cltww" (UID: "7002810c-9381-4adb-95cb-f90d71adb444") : secret "dns-default-metrics-tls" not found
Aug 03 16:04:50.803199 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.803010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:04:50.803199 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.803024    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs podName:50306696-a225-4a3f-b23a-5277b7d4563e nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.803006605 +0000 UTC m=+66.352292900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs") pod "router-default-7646c97b6d-frnzn" (UID: "50306696-a225-4a3f-b23a-5277b7d4563e") : secret "router-metrics-certs-default" not found
Aug 03 16:04:50.803199 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.803060    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 03 16:04:50.803199 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:50.803085    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert podName:3d8c27a8-7bad-47a2-ba46-2ab62d90b198 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:06.803077896 +0000 UTC m=+66.352364190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-wnbm6" (UID: "3d8c27a8-7bad-47a2-ba46-2ab62d90b198") : secret "networking-console-plugin-cert" not found
Aug 03 16:04:50.927084 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:50.927055    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzdx8_17975002-99dd-4015-a156-db842277c1ad/dns-node-resolver/0.log"
Aug 03 16:04:51.789502 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:51.789465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:51.927006 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:51.926973    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xkv75_0ecec175-b462-4ce4-a083-b2192c1b83ab/node-ca/0.log"
Aug 03 16:04:52.326901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:52.326870    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l7gnr_957464d8-2258-4cce-acf5-0d5fecccebd9/serve-healthcheck-canary/0.log"
Aug 03 16:04:52.518950 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:52.518909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:52.519119 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:52.518968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:52.519119 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:52.519073    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:52.519226 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:52.519168    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:04:56.519147078 +0000 UTC m=+56.068433430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:52.519630 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:52.519607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10aead5f-7f93-4830-8756-f30d0e554e5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:52.930180 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:52.930092    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-9qzsb_0c729d6b-549e-4975-87be-65b77ffd61e4/kube-storage-version-migrator-operator/0.log"
Aug 03 16:04:53.487584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:53.486889    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:53.521830 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:53.521799    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-bdhtx"]
Aug 03 16:04:53.525587 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:04:53.525510    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb43105_598e_4db8_a872_f358c152ebb5.slice/crio-1815208b03a02ab71885e0e7934210cb4be550f1def6eb27e0c3372950c2b31d WatchSource:0}: Error finding container 1815208b03a02ab71885e0e7934210cb4be550f1def6eb27e0c3372950c2b31d: Status 404 returned error can't find the container with id 1815208b03a02ab71885e0e7934210cb4be550f1def6eb27e0c3372950c2b31d
Aug 03 16:04:54.343094 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.343057    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k" event={"ID":"d258a910-019a-4e1e-8a08-7e63343e0eba","Type":"ContainerStarted","Data":"6a1e4a4c59e63bdad8d74f53adb47f5e7bc296f06b9f263744bd1dd6b2add6be"}
Aug 03 16:04:54.343598 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.343152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k" event={"ID":"d258a910-019a-4e1e-8a08-7e63343e0eba","Type":"ContainerStarted","Data":"eeb5ba4a15eb63893f8ee494cbe1d63eb08f4676b48c079237b8587d52c6c771"}
Aug 03 16:04:54.345171 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.345140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" event={"ID":"73602750-0eb5-4e17-9b15-43d035f18386","Type":"ContainerStarted","Data":"cd00c98ece9d4b685768a57f4fcf6540db9f585512a37782c57dcdff2bc925ce"}
Aug 03 16:04:54.345291 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.345180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" event={"ID":"73602750-0eb5-4e17-9b15-43d035f18386","Type":"ContainerStarted","Data":"7c212f16a312bcf9bb455ffbe80732c2e4105d79fd777cdf1dfe8569ae21bb3b"}
Aug 03 16:04:54.345291 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.345200    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" event={"ID":"73602750-0eb5-4e17-9b15-43d035f18386","Type":"ContainerStarted","Data":"931427cba0660ade6c593affb7c2f68c41f6def21b95e63c06274147636cf4a6"}
Aug 03 16:04:54.346705 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.346678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg" event={"ID":"fdd8d678-bd20-4277-a67f-ad2a42fcf14c","Type":"ContainerStarted","Data":"b97d4192b69ad8af02502993d888c9493f298438ca88432cdf7fb30c27f165fd"}
Aug 03 16:04:54.346809 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.346796    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:54.348469 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.348350    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx" event={"ID":"3fb43105-598e-4db8-a872-f358c152ebb5","Type":"ContainerStarted","Data":"2c6fc48e471e7b4fb11e82b74ea2b0762546e667257c13612eb3459f2d2cc5a7"}
Aug 03 16:04:54.348469 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.348390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx" event={"ID":"3fb43105-598e-4db8-a872-f358c152ebb5","Type":"ContainerStarted","Data":"1815208b03a02ab71885e0e7934210cb4be550f1def6eb27e0c3372950c2b31d"}
Aug 03 16:04:54.348796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.348772    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg"
Aug 03 16:04:54.349885 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.349864    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb" event={"ID":"522bc6d9-acbc-4605-b3a1-b3ad0671258f","Type":"ContainerStarted","Data":"c6eab13a3d0d50ea8c42fa41a4aeb7b764e6dd1db2cb3b486f22e1256299e975"}
Aug 03 16:04:54.363566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.363512    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ddx7k" podStartSLOduration=2.278783448 podStartE2EDuration="7.363500421s" totalImagesPullingTime="5.084716973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:54.362448247 +0000 UTC m=+53.911734564" watchObservedRunningTime="2026-08-03 16:04:54.363500421 +0000 UTC m=+53.912786737"
Aug 03 16:04:54.379743 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.379695    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d4db7b7fd-vp2nb" podStartSLOduration=3.375070734 podStartE2EDuration="11.379683412s" totalImagesPullingTime="8.004612678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:54.37865077 +0000 UTC m=+53.927937086" watchObservedRunningTime="2026-08-03 16:04:54.379683412 +0000 UTC m=+53.928969727"
Aug 03 16:04:54.395996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.395951    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6fc4bb974b-rlnrg" podStartSLOduration=3.390464034 podStartE2EDuration="11.395939027s" totalImagesPullingTime="8.005474993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:54.394179619 +0000 UTC m=+53.943465932" watchObservedRunningTime="2026-08-03 16:04:54.395939027 +0000 UTC m=+53.945225343"
Aug 03 16:04:54.410038 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.409998    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-bdhtx" podStartSLOduration=5.409985559 podStartE2EDuration="5.409985559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:54.409231141 +0000 UTC m=+53.958517455" watchObservedRunningTime="2026-08-03 16:04:54.409985559 +0000 UTC m=+53.959271874"
Aug 03 16:04:54.430573 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:54.430508    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" podStartSLOduration=3.5122061479999998 podStartE2EDuration="11.430489287s" totalImagesPullingTime="7.918283139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:54.428837927 +0000 UTC m=+53.978124244" watchObservedRunningTime="2026-08-03 16:04:54.430489287 +0000 UTC m=+53.979775604"
Aug 03 16:04:55.107415 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.107357    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:55.107715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.107441    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:04:55.107923 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.107907    2560 scope.go:117] "RemoveContainer" containerID="6af8dbb52b4105a46cb01bcf9ec503c2ed7db9ddb31b43932e071471a4d9d362"
Aug 03 16:04:55.108138 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:55.108105    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-4xfp9_openshift-console-operator(3181b3d5-81de-4ea5-984c-d83d965219cc)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" podUID="3181b3d5-81de-4ea5-984c-d83d965219cc"
Aug 03 16:04:55.118127 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.118081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:55.215589 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.215556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:55.315867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.315836    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:55.416984 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.416876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:55.482789 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.482736    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:55.628790 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:55.628742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:56.559444 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:56.559402    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:04:56.559843 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:56.559555    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:56.559843 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:04:56.559645    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls podName:10aead5f-7f93-4830-8756-f30d0e554e5c nodeName:}" failed. No retries permitted until 2026-08-03 16:05:04.559631705 +0000 UTC m=+64.108918003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ld9nn" (UID: "10aead5f-7f93-4830-8756-f30d0e554e5c") : secret "insights-runtime-extractor-tls" not found
Aug 03 16:04:56.699334 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:56.699287    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:56.753265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:56.753229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:57.015824 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:57.015739    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:57.207005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:57.206966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:57.379178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:57.379137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:57.463101 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:57.463073    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:57.607553 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:57.607496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:57.811477 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:57.811447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:58.811239 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:58.811190    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:59.009061 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:59.009034    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:59.109802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:59.109724    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:59.238041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:59.238004    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:59.411095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:59.411013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:04:59.611930 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:04:59.611898    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:00.013594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:00.013564    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:00.067968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:00.067930    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:00.623777 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:00.623743    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:00.723832 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:00.723805    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:00.824328 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:00.824297    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:01.007949 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.007868    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:01.202031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.201994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:01.244601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.244571    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-flzt6"
Aug 03 16:05:01.443142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.443116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:01.508122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.508087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:01.590617 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.590584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:01.868047 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:01.868013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:02.216836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:02.216761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:02.810173 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:02.810141    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:03.014044 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:03.014014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:03.420216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:03.420179    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:03.612077 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:03.612042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:03.774931 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:03.774827    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" podUID="73602750-0eb5-4e17-9b15-43d035f18386" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 03 16:05:03.813609 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:03.813581    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:04.634150 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:04.634109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:05:04.636398 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:04.636379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10aead5f-7f93-4830-8756-f30d0e554e5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ld9nn\" (UID: \"10aead5f-7f93-4830-8756-f30d0e554e5c\") " pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:05:04.719373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:04.719340    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-grgrw\""
Aug 03 16:05:04.727477 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:04.727454    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ld9nn"
Aug 03 16:05:04.887509 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:04.887482    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ld9nn"]
Aug 03 16:05:04.889442 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:04.889406    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10aead5f_7f93_4830_8756_f30d0e554e5c.slice/crio-467ab4c7d0d66cfdfbae8663e4becf20c3277d092165b2b8cec8655640eac227 WatchSource:0}: Error finding container 467ab4c7d0d66cfdfbae8663e4becf20c3277d092165b2b8cec8655640eac227: Status 404 returned error can't find the container with id 467ab4c7d0d66cfdfbae8663e4becf20c3277d092165b2b8cec8655640eac227
Aug 03 16:05:05.014503 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.014371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:05.222143 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.222060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:05.388807 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.388770    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ld9nn" event={"ID":"10aead5f-7f93-4830-8756-f30d0e554e5c","Type":"ContainerStarted","Data":"8ec6a44777e71b2f63d7c5b848c9d1e62038a46f37a8bfdfeea795ab050332ec"}
Aug 03 16:05:05.388965 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.388828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ld9nn" event={"ID":"10aead5f-7f93-4830-8756-f30d0e554e5c","Type":"ContainerStarted","Data":"467ab4c7d0d66cfdfbae8663e4becf20c3277d092165b2b8cec8655640eac227"}
Aug 03 16:05:05.412900 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.412872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:05.613040 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.613005    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:05.642621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.642598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:05:05.645466 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.645447    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 03 16:05:05.655708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.655688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd3d71ba-13b7-41e1-a0ab-095fd1f4509c-original-pull-secret\") pod \"global-pull-secret-syncer-zkvxk\" (UID: \"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c\") " pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:05:05.706231 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.706203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkvxk"
Aug 03 16:05:05.743455 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.743427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:05:05.746286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.746266    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 03 16:05:05.755768 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.755743    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3749bbe7-77cb-4a21-a66e-e9067b80b42f-metrics-certs\") pod \"network-metrics-daemon-qmv94\" (UID: \"3749bbe7-77cb-4a21-a66e-e9067b80b42f\") " pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:05:05.817264 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.817235    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:05.843973 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.843942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:05:05.846352 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.846327    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9f9k\" (UniqueName: \"kubernetes.io/projected/ba9f5fca-83a4-45ad-a17d-8e49f27205e6-kube-api-access-h9f9k\") pod \"network-check-target-mdnkf\" (UID: \"ba9f5fca-83a4-45ad-a17d-8e49f27205e6\") " pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:05:05.852708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.852428    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zkvxk"]
Aug 03 16:05:05.855276 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:05.855254    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd3d71ba_13b7_41e1_a0ab_095fd1f4509c.slice/crio-cbf4dd75986c71185a38a6a248df055de0be2b9decba71cdf1a480eb09c7190a WatchSource:0}: Error finding container cbf4dd75986c71185a38a6a248df055de0be2b9decba71cdf1a480eb09c7190a: Status 404 returned error can't find the container with id cbf4dd75986c71185a38a6a248df055de0be2b9decba71cdf1a480eb09c7190a
Aug 03 16:05:05.996005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:05.995927    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xrdkc\""
Aug 03 16:05:06.002866 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.002850    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzpf\""
Aug 03 16:05:06.003891 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.003874    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmv94"
Aug 03 16:05:06.011134 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.011120    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:05:06.181946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.181909    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qmv94"]
Aug 03 16:05:06.184596 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:06.184567    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3749bbe7_77cb_4a21_a66e_e9067b80b42f.slice/crio-967ede6f58c3cb816196af9a27b60fe32ab8a80c64ccd59baa2a2f91ff56ef2c WatchSource:0}: Error finding container 967ede6f58c3cb816196af9a27b60fe32ab8a80c64ccd59baa2a2f91ff56ef2c: Status 404 returned error can't find the container with id 967ede6f58c3cb816196af9a27b60fe32ab8a80c64ccd59baa2a2f91ff56ef2c
Aug 03 16:05:06.215672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.215652    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mdnkf"]
Aug 03 16:05:06.217794 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:06.217767    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9f5fca_83a4_45ad_a17d_8e49f27205e6.slice/crio-1a821fed8ca7783ae2bc4939eefa03a2a8ed9809911569958a55486174d94b06 WatchSource:0}: Error finding container 1a821fed8ca7783ae2bc4939eefa03a2a8ed9809911569958a55486174d94b06: Status 404 returned error can't find the container with id 1a821fed8ca7783ae2bc4939eefa03a2a8ed9809911569958a55486174d94b06
Aug 03 16:05:06.393441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.393410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zkvxk" event={"ID":"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c","Type":"ContainerStarted","Data":"cbf4dd75986c71185a38a6a248df055de0be2b9decba71cdf1a480eb09c7190a"}
Aug 03 16:05:06.395194 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.395161    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mdnkf" event={"ID":"ba9f5fca-83a4-45ad-a17d-8e49f27205e6","Type":"ContainerStarted","Data":"237298ad09d8fc38cb14cb322903667c6fb8c3aed674f8aecf172c8eda48a0b0"}
Aug 03 16:05:06.395323 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.395206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mdnkf" event={"ID":"ba9f5fca-83a4-45ad-a17d-8e49f27205e6","Type":"ContainerStarted","Data":"1a821fed8ca7783ae2bc4939eefa03a2a8ed9809911569958a55486174d94b06"}
Aug 03 16:05:06.395323 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.395255    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:05:06.396376 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.396356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qmv94" event={"ID":"3749bbe7-77cb-4a21-a66e-e9067b80b42f","Type":"ContainerStarted","Data":"967ede6f58c3cb816196af9a27b60fe32ab8a80c64ccd59baa2a2f91ff56ef2c"}
Aug 03 16:05:06.411286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.411241    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mdnkf" podStartSLOduration=65.411225102 podStartE2EDuration="1m5.411225102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:06.410755367 +0000 UTC m=+65.960041685" watchObservedRunningTime="2026-08-03 16:05:06.411225102 +0000 UTC m=+65.960511418"
Aug 03 16:05:06.754103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.754065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:05:06.754468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.754108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:05:06.754468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.754174    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:05:06.756538 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.756494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"image-registry-77dc6b9865-ptnlr\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") " pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:05:06.756637 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.756498    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fcc1d13c-3471-483a-80b1-0b7f726d7da7-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-hzzkq\" (UID: \"fcc1d13c-3471-483a-80b1-0b7f726d7da7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:05:06.756637 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.756607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1c80cef-5998-485d-aacb-7d72d6a9d1fc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-p4b6q\" (UID: \"c1c80cef-5998-485d-aacb-7d72d6a9d1fc\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:05:06.855641 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.855614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:05:06.855782 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.855677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:06.855782 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.855716    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:05:06.855782 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.855745    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:06.856912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.856860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50306696-a225-4a3f-b23a-5277b7d4563e-service-ca-bundle\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:06.858517 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.858494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7002810c-9381-4adb-95cb-f90d71adb444-metrics-tls\") pod \"dns-default-cltww\" (UID: \"7002810c-9381-4adb-95cb-f90d71adb444\") " pod="openshift-dns/dns-default-cltww"
Aug 03 16:05:06.859118 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.859093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50306696-a225-4a3f-b23a-5277b7d4563e-metrics-certs\") pod \"router-default-7646c97b6d-frnzn\" (UID: \"50306696-a225-4a3f-b23a-5277b7d4563e\") " pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:06.859584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.859562    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d8c27a8-7bad-47a2-ba46-2ab62d90b198-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-wnbm6\" (UID: \"3d8c27a8-7bad-47a2-ba46-2ab62d90b198\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:05:06.893048 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.893023    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xchfl\""
Aug 03 16:05:06.900671 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.900648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:05:06.940414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.940367    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s5pl7\""
Aug 03 16:05:06.947850 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.947807    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"
Aug 03 16:05:06.955303 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.955276    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2jdbc\""
Aug 03 16:05:06.963715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.963661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"
Aug 03 16:05:06.981752 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.981725    2560 scope.go:117] "RemoveContainer" containerID="6af8dbb52b4105a46cb01bcf9ec503c2ed7db9ddb31b43932e071471a4d9d362"
Aug 03 16:05:06.991621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.991600    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vz785\""
Aug 03 16:05:06.999967 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:06.999942    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cltww"
Aug 03 16:05:07.010230 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.006799    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8jpsd\""
Aug 03 16:05:07.015008 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.014824    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:07.049796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.047214    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ztcth\""
Aug 03 16:05:07.055513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.055291    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"
Aug 03 16:05:07.085233 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.083611    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77dc6b9865-ptnlr"]
Aug 03 16:05:07.098790 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:07.098762    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbc06ec_cfa9_46ef_abbe_f1b14ec97dff.slice/crio-58c6f7ed4ca45d3176ceb46975aa76444b6aecc3ad3a8d702fc707f81b104a0d WatchSource:0}: Error finding container 58c6f7ed4ca45d3176ceb46975aa76444b6aecc3ad3a8d702fc707f81b104a0d: Status 404 returned error can't find the container with id 58c6f7ed4ca45d3176ceb46975aa76444b6aecc3ad3a8d702fc707f81b104a0d
Aug 03 16:05:07.197202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.197036    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q"]
Aug 03 16:05:07.229098 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.229064    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq"]
Aug 03 16:05:07.283813 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.283738    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cltww"]
Aug 03 16:05:07.298575 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:07.298543    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7002810c_9381_4adb_95cb_f90d71adb444.slice/crio-166d4aaf07302ee6785202931f61ddd49ecc8acae4cd94ff513028a3b9b2a30f WatchSource:0}: Error finding container 166d4aaf07302ee6785202931f61ddd49ecc8acae4cd94ff513028a3b9b2a30f: Status 404 returned error can't find the container with id 166d4aaf07302ee6785202931f61ddd49ecc8acae4cd94ff513028a3b9b2a30f
Aug 03 16:05:07.311141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.310980    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7646c97b6d-frnzn"]
Aug 03 16:05:07.313250 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:07.313201    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50306696_a225_4a3f_b23a_5277b7d4563e.slice/crio-ce5f2775bbc4fba84f302c9030080848622ed0ac710656f2fb230225ca30b060 WatchSource:0}: Error finding container ce5f2775bbc4fba84f302c9030080848622ed0ac710656f2fb230225ca30b060: Status 404 returned error can't find the container with id ce5f2775bbc4fba84f302c9030080848622ed0ac710656f2fb230225ca30b060
Aug 03 16:05:07.331189 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.331126    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6"]
Aug 03 16:05:07.336162 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:07.336126    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d8c27a8_7bad_47a2_ba46_2ab62d90b198.slice/crio-d214971e09058b8b30a64d70b02629abb2ecf324342ba7822fe5d092af44a84d WatchSource:0}: Error finding container d214971e09058b8b30a64d70b02629abb2ecf324342ba7822fe5d092af44a84d: Status 404 returned error can't find the container with id d214971e09058b8b30a64d70b02629abb2ecf324342ba7822fe5d092af44a84d
Aug 03 16:05:07.402742 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.402666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq" event={"ID":"fcc1d13c-3471-483a-80b1-0b7f726d7da7","Type":"ContainerStarted","Data":"2c957248ee72b052c787eb8d8568468514715e4830de0d0d5a7cc4d48f0b4b80"}
Aug 03 16:05:07.406507 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.406108    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:05:07.406507 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.406207    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" event={"ID":"3181b3d5-81de-4ea5-984c-d83d965219cc","Type":"ContainerStarted","Data":"f643efe3838b53cb7a4fc52ebced0a9734f78ae42220f7b5246eca7087167b7b"}
Aug 03 16:05:07.407173 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.407101    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:05:07.411015 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.410882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ld9nn" event={"ID":"10aead5f-7f93-4830-8756-f30d0e554e5c","Type":"ContainerStarted","Data":"e559485fd29ba874a5677409c900a87c6a4614d5ef235cfa942ede854c5605a2"}
Aug 03 16:05:07.413273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.413211    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7646c97b6d-frnzn" event={"ID":"50306696-a225-4a3f-b23a-5277b7d4563e","Type":"ContainerStarted","Data":"ce5f2775bbc4fba84f302c9030080848622ed0ac710656f2fb230225ca30b060"}
Aug 03 16:05:07.415017 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.414989    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q" event={"ID":"c1c80cef-5998-485d-aacb-7d72d6a9d1fc","Type":"ContainerStarted","Data":"af73ca5de391ff492ea54258d2e86ffa1439eafd7151780b6c1cc6b250dfc395"}
Aug 03 16:05:07.417120 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.417060    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6" event={"ID":"3d8c27a8-7bad-47a2-ba46-2ab62d90b198","Type":"ContainerStarted","Data":"d214971e09058b8b30a64d70b02629abb2ecf324342ba7822fe5d092af44a84d"}
Aug 03 16:05:07.418978 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.418917    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cltww" event={"ID":"7002810c-9381-4adb-95cb-f90d71adb444","Type":"ContainerStarted","Data":"166d4aaf07302ee6785202931f61ddd49ecc8acae4cd94ff513028a3b9b2a30f"}
Aug 03 16:05:07.421438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.421260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" event={"ID":"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff","Type":"ContainerStarted","Data":"98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959"}
Aug 03 16:05:07.421565 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.421299    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" event={"ID":"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff","Type":"ContainerStarted","Data":"58c6f7ed4ca45d3176ceb46975aa76444b6aecc3ad3a8d702fc707f81b104a0d"}
Aug 03 16:05:07.421565 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.421562    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:05:07.428250 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.427921    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9" podStartSLOduration=35.698442979 podStartE2EDuration="45.427844696s" totalImagesPullingTime="9.729401717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:07.426030525 +0000 UTC m=+66.975316834" watchObservedRunningTime="2026-08-03 16:05:07.427844696 +0000 UTC m=+66.977131013"
Aug 03 16:05:07.446190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:07.444408    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" podStartSLOduration=66.444388743 podStartE2EDuration="1m6.444388743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:07.444017652 +0000 UTC m=+66.993303969" watchObservedRunningTime="2026-08-03 16:05:07.444388743 +0000 UTC m=+66.993675060"
Aug 03 16:05:08.348672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:08.348641    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-4xfp9"
Aug 03 16:05:08.428260 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:08.428136    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7646c97b6d-frnzn" event={"ID":"50306696-a225-4a3f-b23a-5277b7d4563e","Type":"ContainerStarted","Data":"41a9f16e1ea529760a4b7bd45cb112e1f3031c6cc16080dcd15c26db6ef4b768"}
Aug 03 16:05:08.447745 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:08.447696    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7646c97b6d-frnzn" podStartSLOduration=46.447679631 podStartE2EDuration="46.447679631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:08.446408554 +0000 UTC m=+67.995694865" watchObservedRunningTime="2026-08-03 16:05:08.447679631 +0000 UTC m=+67.996965944"
Aug 03 16:05:09.015833 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.015695    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:09.019009 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.018987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:09.435928 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.435851    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qmv94" event={"ID":"3749bbe7-77cb-4a21-a66e-e9067b80b42f","Type":"ContainerStarted","Data":"16272d226c1019ac4d7c5e44fd35b98f00a3d924aecb0c9b7c6e63b7d38616bd"}
Aug 03 16:05:09.436362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.436321    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:09.437786 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.437759    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7646c97b6d-frnzn"
Aug 03 16:05:09.463008 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.462927    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:09.519144 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.519098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:09.612450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:09.612413    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:10.010650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:10.010565    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:10.111073 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:10.111036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:10.218693 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:10.218656    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:10.273308 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:10.273159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:10.510305 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:10.510164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:12.626325 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:12.626296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:12.683624 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:12.683574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.050029 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.050001    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-wx647"]
Aug 03 16:05:13.053010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.052984    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.059646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.059626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:13.062205 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.062186    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 03 16:05:13.062324 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.062251    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-w2kkn\""
Aug 03 16:05:13.062324 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.062293    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 03 16:05:13.068496 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.068472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-wx647"]
Aug 03 16:05:13.148239 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.148205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.217675 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.217644    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.222631 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.222607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nrk8\" (UniqueName: \"kubernetes.io/projected/92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17-kube-api-access-2nrk8\") pod \"downloads-77c974ffcf-wx647\" (UID: \"92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17\") " pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:13.315115 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.315079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.323884 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.323829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nrk8\" (UniqueName: \"kubernetes.io/projected/92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17-kube-api-access-2nrk8\") pod \"downloads-77c974ffcf-wx647\" (UID: \"92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17\") " pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:13.334291 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.334266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nrk8\" (UniqueName: \"kubernetes.io/projected/92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17-kube-api-access-2nrk8\") pod \"downloads-77c974ffcf-wx647\" (UID: \"92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17\") " pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:13.369178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.369153    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:13.737270 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.737201    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.774040 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.774000    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" podUID="73602750-0eb5-4e17-9b15-43d035f18386" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 03 16:05:13.837926 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.837885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:13.977794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:13.977762    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:14.078355 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.078331    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:14.242462 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.242289    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-wx647"]
Aug 03 16:05:14.247680 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:14.247592    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a1c9e2_aad4_47fb_8b4f_4ff3bc520e17.slice/crio-2b18ebe73b34017a608ca38b51114cb68d9048c85495f35dc85b69ab23fca50d WatchSource:0}: Error finding container 2b18ebe73b34017a608ca38b51114cb68d9048c85495f35dc85b69ab23fca50d: Status 404 returned error can't find the container with id 2b18ebe73b34017a608ca38b51114cb68d9048c85495f35dc85b69ab23fca50d
Aug 03 16:05:14.467965 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.467855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zkvxk" event={"ID":"cd3d71ba-13b7-41e1-a0ab-095fd1f4509c","Type":"ContainerStarted","Data":"93f51a60615be3691e9f698993a170c5a72e7c5e827ee3471cf0761732e290a4"}
Aug 03 16:05:14.470118 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.470090    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq" event={"ID":"fcc1d13c-3471-483a-80b1-0b7f726d7da7","Type":"ContainerStarted","Data":"e3edeb4985722c88253db7880eaeff6456d2ef8fdc4111982152ad82fde63e86"}
Aug 03 16:05:14.472907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.472875    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ld9nn" event={"ID":"10aead5f-7f93-4830-8756-f30d0e554e5c","Type":"ContainerStarted","Data":"10febe9da28f5068f69595a2ad16fc150b3c15cb00e7968bdedfe2f4c7ca3f2b"}
Aug 03 16:05:14.476814 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.476234    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qmv94" event={"ID":"3749bbe7-77cb-4a21-a66e-e9067b80b42f","Type":"ContainerStarted","Data":"c1942c9812b74cd93a03ab498d62cc78149de05b575288e9079ae1d89e3222dd"}
Aug 03 16:05:14.479178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.479151    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q" event={"ID":"c1c80cef-5998-485d-aacb-7d72d6a9d1fc","Type":"ContainerStarted","Data":"a2accc45564a37007ae1b615d12de2f1a0e06b9e7ba201b7ec587b9d6326ed2d"}
Aug 03 16:05:14.481413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.481391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6" event={"ID":"3d8c27a8-7bad-47a2-ba46-2ab62d90b198","Type":"ContainerStarted","Data":"ce6401994ba4287864bb38ad5f892798aaf83ab75d4c765aabd8cb4282a46abf"}
Aug 03 16:05:14.483645 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.483597    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zkvxk" podStartSLOduration=65.251446762 podStartE2EDuration="1m13.483580359s" totalImagesPullingTime="8.232133597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:14.482484866 +0000 UTC m=+74.031771184" watchObservedRunningTime="2026-08-03 16:05:14.483580359 +0000 UTC m=+74.032866675"
Aug 03 16:05:14.484885 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.484868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cltww" event={"ID":"7002810c-9381-4adb-95cb-f90d71adb444","Type":"ContainerStarted","Data":"fb43f6a932fc59dc6e5b16d6a48108657848829c1c1804939fc8c08ab754232c"}
Aug 03 16:05:14.486187 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.486170    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-wx647" event={"ID":"92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17","Type":"ContainerStarted","Data":"2b18ebe73b34017a608ca38b51114cb68d9048c85495f35dc85b69ab23fca50d"}
Aug 03 16:05:14.501925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.501883    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qmv94" podStartSLOduration=70.592568851 podStartE2EDuration="1m13.501863629s" totalImagesPullingTime="2.909294778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:14.499372052 +0000 UTC m=+74.048658371" watchObservedRunningTime="2026-08-03 16:05:14.501863629 +0000 UTC m=+74.051149945"
Aug 03 16:05:14.528100 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.528024    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ld9nn" podStartSLOduration=17.603770255 podStartE2EDuration="26.528002926s" totalImagesPullingTime="8.924232671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:14.526026177 +0000 UTC m=+74.075312492" watchObservedRunningTime="2026-08-03 16:05:14.528002926 +0000 UTC m=+74.077289243"
Aug 03 16:05:14.547089 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.546687    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-wnbm6" podStartSLOduration=45.796250589 podStartE2EDuration="52.546669271s" totalImagesPullingTime="6.750418682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:14.546422019 +0000 UTC m=+74.095708336" watchObservedRunningTime="2026-08-03 16:05:14.546669271 +0000 UTC m=+74.095955588"
Aug 03 16:05:14.571122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.571064    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-p4b6q" podStartSLOduration=45.685852849 podStartE2EDuration="52.571041931s" totalImagesPullingTime="6.885189082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:14.570101831 +0000 UTC m=+74.119388148" watchObservedRunningTime="2026-08-03 16:05:14.571041931 +0000 UTC m=+74.120328248"
Aug 03 16:05:14.627695 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:14.627660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:15.492214 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.492179    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cltww" event={"ID":"7002810c-9381-4adb-95cb-f90d71adb444","Type":"ContainerStarted","Data":"690a96b287892ca061862ef0b16d1fb1e721242b1ca6e801d676a1ebfd39dc4b"}
Aug 03 16:05:15.492658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.492285    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cltww"
Aug 03 16:05:15.493984 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.493959    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq" event={"ID":"fcc1d13c-3471-483a-80b1-0b7f726d7da7","Type":"ContainerStarted","Data":"1a0206ac3ec48edc443ace53109d724e1e1e61a304f6dfc3aebbf8f5f9dcea2c"}
Aug 03 16:05:15.511219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.511177    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cltww" podStartSLOduration=34.721086156 podStartE2EDuration="41.51116308s" totalImagesPullingTime="6.790076924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:15.508856394 +0000 UTC m=+75.058142711" watchObservedRunningTime="2026-08-03 16:05:15.51116308 +0000 UTC m=+75.060449396"
Aug 03 16:05:15.525332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.525292    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-hzzkq" podStartSLOduration=46.740525612 podStartE2EDuration="53.525281983s" totalImagesPullingTime="6.784756371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:15.524171806 +0000 UTC m=+75.073458149" watchObservedRunningTime="2026-08-03 16:05:15.525281983 +0000 UTC m=+75.074568299"
Aug 03 16:05:15.559959 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.559919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:15.615352 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:15.615314    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:16.667628 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:16.667603    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:16.768923 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:16.768890    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:18.520296 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:18.520259    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:18.620840 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:18.620802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:19.811993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:19.811943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:19.871716 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:19.871682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:20.167270 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:20.167192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:21.441027 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:21.440994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:21.515481 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:21.515443    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:21.585328 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:21.585291    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:23.291513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:23.291448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:23.774413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:23.774332    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" podUID="73602750-0eb5-4e17-9b15-43d035f18386" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 03 16:05:23.774413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:23.774407    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n"
Aug 03 16:05:23.774897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:23.774867    2560 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"cd00c98ece9d4b685768a57f4fcf6540db9f585512a37782c57dcdff2bc925ce"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Aug 03 16:05:23.774943 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:23.774917    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" podUID="73602750-0eb5-4e17-9b15-43d035f18386" containerName="service-proxy" containerID="cri-o://cd00c98ece9d4b685768a57f4fcf6540db9f585512a37782c57dcdff2bc925ce" gracePeriod=30
Aug 03 16:05:24.047697 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:24.047650    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:24.538623 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:24.538589    2560 generic.go:356] "Generic (PLEG): container finished" podID="73602750-0eb5-4e17-9b15-43d035f18386" containerID="cd00c98ece9d4b685768a57f4fcf6540db9f585512a37782c57dcdff2bc925ce" exitCode=2
Aug 03 16:05:24.539117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:24.538677    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" event={"ID":"73602750-0eb5-4e17-9b15-43d035f18386","Type":"ContainerDied","Data":"cd00c98ece9d4b685768a57f4fcf6540db9f585512a37782c57dcdff2bc925ce"}
Aug 03 16:05:24.539117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:24.538737    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-759dd77cff-qjv5n" event={"ID":"73602750-0eb5-4e17-9b15-43d035f18386","Type":"ContainerStarted","Data":"ac0f40f0a58fc2152b11e6358000e1d8b8a2b95adc35f4e4a9014107666d0e6d"}
Aug 03 16:05:25.242595 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.242558    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p646h"]
Aug 03 16:05:25.246764 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.246745    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.249550 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.249503    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 03 16:05:25.249550 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.249517    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 03 16:05:25.249817 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.249802    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 03 16:05:25.250746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.250719    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8f8d9\""
Aug 03 16:05:25.250850 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.250719    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 03 16:05:25.317714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.317714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317689    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/172b4ca6-9e01-4f1e-9386-83ea186d3b56-metrics-client-ca\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.317714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-wtmp\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.318016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-sys\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.318016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-tls\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.318016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-textfile\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.318016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-accelerators-collector-config\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.318016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh9jv\" (UniqueName: \"kubernetes.io/projected/172b4ca6-9e01-4f1e-9386-83ea186d3b56-kube-api-access-hh9jv\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.318016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.317994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-root\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.418613 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.418553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-sys\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.418613 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.418610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-tls\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.418836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.418664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-textfile\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.418836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.418692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-accelerators-collector-config\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.418925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.418859    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-sys\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419006 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.418980    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh9jv\" (UniqueName: \"kubernetes.io/projected/172b4ca6-9e01-4f1e-9386-83ea186d3b56-kube-api-access-hh9jv\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419129 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419012    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-root\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419129 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419129 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419116    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-root\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419300 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419161    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/172b4ca6-9e01-4f1e-9386-83ea186d3b56-metrics-client-ca\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419300 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419191    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-wtmp\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419300 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419275    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-textfile\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419445 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-wtmp\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-accelerators-collector-config\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.419841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.419801    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/172b4ca6-9e01-4f1e-9386-83ea186d3b56-metrics-client-ca\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.421841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.421816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-tls\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.421985 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.421966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/172b4ca6-9e01-4f1e-9386-83ea186d3b56-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.427667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.427642    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh9jv\" (UniqueName: \"kubernetes.io/projected/172b4ca6-9e01-4f1e-9386-83ea186d3b56-kube-api-access-hh9jv\") pod \"node-exporter-p646h\" (UID: \"172b4ca6-9e01-4f1e-9386-83ea186d3b56\") " pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.500748 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.500665    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cltww"
Aug 03 16:05:25.558832 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.558795    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p646h"
Aug 03 16:05:25.577950 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:05:25.577826    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172b4ca6_9e01_4f1e_9386_83ea186d3b56.slice/crio-c908262899472a177c1711864a7e3042c1dbcc4d6a39b6e03852e849cad01469 WatchSource:0}: Error finding container c908262899472a177c1711864a7e3042c1dbcc4d6a39b6e03852e849cad01469: Status 404 returned error can't find the container with id c908262899472a177c1711864a7e3042c1dbcc4d6a39b6e03852e849cad01469
Aug 03 16:05:25.835961 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.835921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:25.897922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:25.897883    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:26.183010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.182702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:26.276310 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.276266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:26.557761 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.557718    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p646h" event={"ID":"172b4ca6-9e01-4f1e-9386-83ea186d3b56","Type":"ContainerStarted","Data":"c908262899472a177c1711864a7e3042c1dbcc4d6a39b6e03852e849cad01469"}
Aug 03 16:05:26.692966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.692861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:26.792429 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.792343    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:26.906505 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.906409    2560 patch_prober.go:28] interesting pod/image-registry-77dc6b9865-ptnlr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 03 16:05:26.906505 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:26.906470    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" podUID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 03 16:05:27.635099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:27.635064    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:27.735446 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:27.735410    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:28.165232 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:28.164937    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:28.260441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:28.260407    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:28.360075 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:28.360027    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:28.433391 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:28.433235    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:05:28.716749 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:28.716663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:28.815439 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:28.815403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:29.635424 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:29.635377    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:29.695400 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:29.695320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:29.969421 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:29.969329    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:30.036810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:30.036772    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:30.692450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:30.692413    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:31.419376 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:31.419334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:31.480951 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:31.480881    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:32.242583 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:32.242546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:32.304169 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:32.304133    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:33.034422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:33.034388    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:33.598680 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:33.595801    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-wx647" event={"ID":"92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17","Type":"ContainerStarted","Data":"c31c1a5d75b4877427f02e889addf106125931631fa6d5e96ab2b5e976cfe178"}
Aug 03 16:05:33.598680 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:33.597206    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:33.608800 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:33.608766    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-wx647"
Aug 03 16:05:33.634839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:33.633013    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-wx647" podStartSLOduration=1.749389124 podStartE2EDuration="20.632989584s" totalImagesPullingTime="18.88360046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:33.617077335 +0000 UTC m=+93.166363653" watchObservedRunningTime="2026-08-03 16:05:33.632989584 +0000 UTC m=+93.182275902"
Aug 03 16:05:33.903469 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:33.903430    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:34.609920 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:34.609884    2560 generic.go:356] "Generic (PLEG): container finished" podID="172b4ca6-9e01-4f1e-9386-83ea186d3b56" containerID="9a60227c01a5413d224c46e965fdb3e544dfdbdc48bba27d85082a9e820b4904" exitCode=0
Aug 03 16:05:34.610409 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:34.609983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p646h" event={"ID":"172b4ca6-9e01-4f1e-9386-83ea186d3b56","Type":"ContainerDied","Data":"9a60227c01a5413d224c46e965fdb3e544dfdbdc48bba27d85082a9e820b4904"}
Aug 03 16:05:35.416695 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.416654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:35.519416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.519380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:35.585993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.585956    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77dc6b9865-ptnlr"]
Aug 03 16:05:35.617491 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.617439    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p646h" event={"ID":"172b4ca6-9e01-4f1e-9386-83ea186d3b56","Type":"ContainerStarted","Data":"107d0f702b437a0cc7a8096a629e5e4eeb0f840f98ed64bf9ed4e867793935b9"}
Aug 03 16:05:35.617941 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.617507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p646h" event={"ID":"172b4ca6-9e01-4f1e-9386-83ea186d3b56","Type":"ContainerStarted","Data":"75d56638fdc404240cc3ee3ff6e73a09367e8655c3b1705167e96df0e4904dd9"}
Aug 03 16:05:35.635982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.635911    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p646h" podStartSLOduration=2.356807775 podStartE2EDuration="10.635890473s" totalImagesPullingTime="8.198305576s" totalInitContainerRuntime="80.777122ms" isStatefulPod=true podCreationTimestamp="2026-08-03 16:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:35.635406644 +0000 UTC m=+95.184692975" watchObservedRunningTime="2026-08-03 16:05:35.635890473 +0000 UTC m=+95.185176791"
Aug 03 16:05:35.670335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.670236    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:35.769598 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:35.769558    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:37.424107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:37.424074    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mdnkf"
Aug 03 16:05:37.743643 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:37.743550    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:37.831569 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:37.831513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:39.128095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:39.128060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:39.232440 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:39.232409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:43.677184 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:43.677148    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:43.777479 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:43.777448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:43.878741 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:43.878709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:46.391762 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:46.391729    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:46.451347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:46.451307    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:46.817106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:46.817074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:46.882345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:46.882314    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:54.707134 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:54.707084    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:54.808228 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:54.808195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:54.864667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:54.864633    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:05:56.701584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:56.701486    2560 generic.go:356] "Generic (PLEG): container finished" podID="0c729d6b-549e-4975-87be-65b77ffd61e4" containerID="5e11eea88231cc5cec7cf75be57369c6e170d330cd6b82179ad4bfcbb85a60d1" exitCode=0
Aug 03 16:05:56.701584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:56.701564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb" event={"ID":"0c729d6b-549e-4975-87be-65b77ffd61e4","Type":"ContainerDied","Data":"5e11eea88231cc5cec7cf75be57369c6e170d330cd6b82179ad4bfcbb85a60d1"}
Aug 03 16:05:56.701976 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:56.701872    2560 scope.go:117] "RemoveContainer" containerID="5e11eea88231cc5cec7cf75be57369c6e170d330cd6b82179ad4bfcbb85a60d1"
Aug 03 16:05:57.707675 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:05:57.707639    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-9qzsb" event={"ID":"0c729d6b-549e-4975-87be-65b77ffd61e4","Type":"ContainerStarted","Data":"836097e3809e48f05ee36d4d6af7e5a837d4d7862be843c3854864228b87eeba"}
Aug 03 16:06:00.610093 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:00.610048    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" podUID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" containerName="registry" containerID="cri-o://98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959" gracePeriod=30
Aug 03 16:06:00.876743 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:00.876675    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:06:01.034856 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.034811    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-image-registry-private-configuration\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.034895    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-bound-sa-token\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.034914    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-trusted-ca\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.034937    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-installation-pull-secrets\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.034960    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdb7x\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-kube-api-access-bdb7x\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.034986    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035321 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.035192    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-certificates\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035321 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.035242    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-ca-trust-extracted\") pod \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\" (UID: \"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff\") "
Aug 03 16:06:01.035601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.035563    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:01.035761 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.035730    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:06:01.037591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.037565    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:06:01.037696 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.037600    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:01.037760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.037738    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-kube-api-access-bdb7x" (OuterVolumeSpecName: "kube-api-access-bdb7x") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "kube-api-access-bdb7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:06:01.037803 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.037755    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:06:01.037803 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.037782    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:06:01.046366 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.046343    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" (UID: "adbc06ec-cfa9-46ef-abbe-f1b14ec97dff"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136423    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-bound-sa-token\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136461    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-trusted-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136470    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-installation-pull-secrets\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136479    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdb7x\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-kube-api-access-bdb7x\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136489    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136498    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-registry-certificates\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136505    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-ca-trust-extracted\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.136516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.136515    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff-image-registry-private-configuration\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:06:01.386217 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.386172    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:01.487343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.487247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:01.724302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.724273    2560 generic.go:356] "Generic (PLEG): container finished" podID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" containerID="98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959" exitCode=0
Aug 03 16:06:01.724759 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.724318    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr"
Aug 03 16:06:01.724759 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.724318    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" event={"ID":"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff","Type":"ContainerDied","Data":"98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959"}
Aug 03 16:06:01.724759 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.724449    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77dc6b9865-ptnlr" event={"ID":"adbc06ec-cfa9-46ef-abbe-f1b14ec97dff","Type":"ContainerDied","Data":"58c6f7ed4ca45d3176ceb46975aa76444b6aecc3ad3a8d702fc707f81b104a0d"}
Aug 03 16:06:01.724759 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.724475    2560 scope.go:117] "RemoveContainer" containerID="98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959"
Aug 03 16:06:01.739318 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.739298    2560 scope.go:117] "RemoveContainer" containerID="98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959"
Aug 03 16:06:01.739632 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:06:01.739607    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959\": container with ID starting with 98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959 not found: ID does not exist" containerID="98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959"
Aug 03 16:06:01.739705 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.739644    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959"} err="failed to get container status \"98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959\": rpc error: code = NotFound desc = could not find container \"98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959\": container with ID starting with 98c82df8b80bb35e4ad1f0bf2715c2372db28943e2fa0b12943e4645cda3c959 not found: ID does not exist"
Aug 03 16:06:01.747111 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.747092    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77dc6b9865-ptnlr"]
Aug 03 16:06:01.753412 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:01.753393    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77dc6b9865-ptnlr"]
Aug 03 16:06:02.986198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:02.986164    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" path="/var/lib/kubelet/pods/adbc06ec-cfa9-46ef-abbe-f1b14ec97dff/volumes"
Aug 03 16:06:05.557121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:05.557079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:06.745414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:06.745386    2560 generic.go:356] "Generic (PLEG): container finished" podID="72977dd4-4353-44c8-a984-7b5c0445c7d0" containerID="2ae4a204c1332a919fdde27955633cf0a4d1d10871be0bcd29a9b30419e4101d" exitCode=0
Aug 03 16:06:06.745826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:06.745449    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5" event={"ID":"72977dd4-4353-44c8-a984-7b5c0445c7d0","Type":"ContainerDied","Data":"2ae4a204c1332a919fdde27955633cf0a4d1d10871be0bcd29a9b30419e4101d"}
Aug 03 16:06:06.745826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:06.745741    2560 scope.go:117] "RemoveContainer" containerID="2ae4a204c1332a919fdde27955633cf0a4d1d10871be0bcd29a9b30419e4101d"
Aug 03 16:06:07.750549 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:07.750500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jjz5" event={"ID":"72977dd4-4353-44c8-a984-7b5c0445c7d0","Type":"ContainerStarted","Data":"21e050134c2aa055756339470a0919cf946bf238c7c75770c73ca2f678b77405"}
Aug 03 16:06:17.797118 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:17.797076    2560 generic.go:356] "Generic (PLEG): container finished" podID="12a79723-b930-4937-9c84-ac2f2b437efb" containerID="7898f752971541ee2581f7ab1783c0b498b086f1e1719c068908fdda7aace035" exitCode=0
Aug 03 16:06:17.797561 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:17.797151    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-rt655" event={"ID":"12a79723-b930-4937-9c84-ac2f2b437efb","Type":"ContainerDied","Data":"7898f752971541ee2581f7ab1783c0b498b086f1e1719c068908fdda7aace035"}
Aug 03 16:06:17.797561 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:17.797487    2560 scope.go:117] "RemoveContainer" containerID="7898f752971541ee2581f7ab1783c0b498b086f1e1719c068908fdda7aace035"
Aug 03 16:06:18.803207 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:18.803173    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-rt655" event={"ID":"12a79723-b930-4937-9c84-ac2f2b437efb","Type":"ContainerStarted","Data":"b9fe3fecc6327687e184c673d41fc3cd6b8fc2262921c48d9fe9a4d6d2e915ff"}
Aug 03 16:06:37.341804 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:37.341763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:44.427103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:44.427069    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:44.631256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:44.631221    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:52.966114 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:52.966073    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:06:53.066365 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:06:53.066323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:04.018353 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:04.018315    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:04.113850 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:04.113802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:04.239086 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:04.239050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:49.058396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:49.058357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:49.136281 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:49.136234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:59.643392 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:59.643360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:07:59.702002 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:07:59.701968    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:03.999925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:03.999892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:05.069090 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:05.069061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:05.143869 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:05.143830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:05.223514 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:05.223484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:05.297935 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:05.297905    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:05.815089 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:05.815054    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:05.884953 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:05.884919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:11.715099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:11.715056    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:11.815514 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:11.815473    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:16.292656 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:16.292623    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:16.391836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:16.391802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:16.493366 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:16.493330    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:16.804783 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:16.804751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:16.864490 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:16.864457    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:08:20.682043 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:08:20.682007    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:00.870494 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:00.870465    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:09:00.874878 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:00.874855    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:09:00.880241 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:00.880216    2560 kubelet.go:1648] "Image garbage collection succeeded"
Aug 03 16:09:02.689060 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:02.689024    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:02.787386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:02.787350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:29.504235 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:29.504198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:31.554645 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:31.554606    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:33.101654 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:33.101621    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:33.202935 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:33.202904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:34.779159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:34.779124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:34.878136 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:34.878083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:34.984715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:34.984673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:37.220378 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:37.220343    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:37.418041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:37.418001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:39.901177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:39.901042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:40.000022 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:39.999988    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:44.812282 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:44.812250    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:47.576375 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:47.576335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:47.676944 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:47.676912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:55.218975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:55.218898    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:55.328251 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:55.328220    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:57.451557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:57.451497    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:58.615545 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:58.615496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:09:58.672474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:09:58.672442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:10.134203 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:10.134172    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:10.242860 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:10.242826    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:10.736125 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:10.736087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:10.935309 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:10.935275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.032448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.032365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.079557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.079517    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"]
Aug 03 16:10:11.079831 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.079820    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" containerName="registry"
Aug 03 16:10:11.079875 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.079833    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" containerName="registry"
Aug 03 16:10:11.079908 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.079900    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="adbc06ec-cfa9-46ef-abbe-f1b14ec97dff" containerName="registry"
Aug 03 16:10:11.082890 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.082874    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.085746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.085727    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6rvp9\""
Aug 03 16:10:11.086010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.085996    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 03 16:10:11.086256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.086241    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 03 16:10:11.086314 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.086272    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 03 16:10:11.086424 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.086405    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 03 16:10:11.098792 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.098765    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.098899 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.098833    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x45tt\" (UniqueName: \"kubernetes.io/projected/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-kube-api-access-x45tt\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.098899 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.098860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.098986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.098898    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.099023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.098982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.103122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.103097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"]
Aug 03 16:10:11.199470 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.199429    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.199981 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.199493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.199981 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.199562    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x45tt\" (UniqueName: \"kubernetes.io/projected/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-kube-api-access-x45tt\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.199981 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.199592    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.199981 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.199624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.200139 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.199989    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.201928 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.201903    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.202029 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.201966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.202073 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.202041    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.210276 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.210250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x45tt\" (UniqueName: \"kubernetes.io/projected/f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb-kube-api-access-x45tt\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-48277\" (UID: \"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.274995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.274948    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.375151 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.375065    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.392670 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.392645    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:11.477737 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.477654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.531577 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.531554    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"]
Aug 03 16:10:11.535117 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:10:11.535086    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71ed6ff_8851_4538_ba1c_cdbd36fcf4bb.slice/crio-589340e800f6ef613ae31b3c0bb4bdf23ee19e940d95aa5c04a414d7eb4a866c WatchSource:0}: Error finding container 589340e800f6ef613ae31b3c0bb4bdf23ee19e940d95aa5c04a414d7eb4a866c: Status 404 returned error can't find the container with id 589340e800f6ef613ae31b3c0bb4bdf23ee19e940d95aa5c04a414d7eb4a866c
Aug 03 16:10:11.536973 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.536954    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:10:11.575031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.575000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.703186 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.703099    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277" event={"ID":"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb","Type":"ContainerStarted","Data":"589340e800f6ef613ae31b3c0bb4bdf23ee19e940d95aa5c04a414d7eb4a866c"}
Aug 03 16:10:11.846639 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.846608    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:11.968776 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:11.968688    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:15.724369 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:15.724331    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277" event={"ID":"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb","Type":"ContainerStarted","Data":"8d45b97b383623e0b98214fa4eeda81071f723adc5d718031882db82d67d9154"}
Aug 03 16:10:16.729520 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:16.729482    2560 generic.go:356] "Generic (PLEG): container finished" podID="f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb" containerID="8d45b97b383623e0b98214fa4eeda81071f723adc5d718031882db82d67d9154" exitCode=0
Aug 03 16:10:16.729913 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:16.729566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277" event={"ID":"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb","Type":"ContainerDied","Data":"8d45b97b383623e0b98214fa4eeda81071f723adc5d718031882db82d67d9154"}
Aug 03 16:10:17.736075 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:17.735987    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277" event={"ID":"f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb","Type":"ContainerStarted","Data":"1d17266f89b3753140b72c81abd427004d8faad2a81ba1c2a8bee4d013cb1e80"}
Aug 03 16:10:17.736512 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:17.736130    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:17.758355 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:17.758311    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277" podStartSLOduration=1.851797194 podStartE2EDuration="6.758295774s" totalImagesPullingTime="3.867658535s" totalInitContainerRuntime="1.038840045s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:17.75788723 +0000 UTC m=+377.307173546" watchObservedRunningTime="2026-08-03 16:10:17.758295774 +0000 UTC m=+377.307582091"
Aug 03 16:10:22.976738 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:22.976700    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:23.070000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:23.069969    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:25.339180 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:25.339148    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:25.439165 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:25.439124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:27.230007 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:27.229978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:27.400685 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:27.400641    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:27.468877 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:27.468842    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:28.743015 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:28.742985    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-48277"
Aug 03 16:10:30.473014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:30.472987    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:30.569827 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:30.569782    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:31.197995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:31.197945    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:31.284205 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:31.284157    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:31.357010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:31.356965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:31.456966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:31.456857    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:31.893346 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:31.893297    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:32.007486 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:32.007448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:32.107975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:32.107943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:38.027386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:38.027353    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:38.128760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:38.128723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:40.979831 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:40.979784    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:41.035332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.035301    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"]
Aug 03 16:10:41.044035 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.044009    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.047220 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.047184    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 03 16:10:41.047777 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.047752    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-gr2tz\""
Aug 03 16:10:41.049049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.049024    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"]
Aug 03 16:10:41.054468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.054401    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"]
Aug 03 16:10:41.057937 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.057917    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.060784 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.060762    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-l5xkg\""
Aug 03 16:10:41.060925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.060904    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 03 16:10:41.075610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.075587    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"]
Aug 03 16:10:41.103794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.103761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:41.148192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.148152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f544be1b-e7ba-4bdf-a53e-e6323944b47f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzh7x\" (UID: \"f544be1b-e7ba-4bdf-a53e-e6323944b47f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.148356 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.148217    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2rf\" (UniqueName: \"kubernetes.io/projected/f544be1b-e7ba-4bdf-a53e-e6323944b47f-kube-api-access-qf2rf\") pod \"llmisvc-controller-manager-777c6bd7d5-xzh7x\" (UID: \"f544be1b-e7ba-4bdf-a53e-e6323944b47f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.148356 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.148266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szqh9\" (UniqueName: \"kubernetes.io/projected/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-kube-api-access-szqh9\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.148461 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.148364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.199142 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.199111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:41.249050 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.248948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.249050 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.249006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f544be1b-e7ba-4bdf-a53e-e6323944b47f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzh7x\" (UID: \"f544be1b-e7ba-4bdf-a53e-e6323944b47f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.249050 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.249051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2rf\" (UniqueName: \"kubernetes.io/projected/f544be1b-e7ba-4bdf-a53e-e6323944b47f-kube-api-access-qf2rf\") pod \"llmisvc-controller-manager-777c6bd7d5-xzh7x\" (UID: \"f544be1b-e7ba-4bdf-a53e-e6323944b47f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.249358 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.249086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szqh9\" (UniqueName: \"kubernetes.io/projected/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-kube-api-access-szqh9\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.249358 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:10:41.249098    2560 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Aug 03 16:10:41.249358 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:10:41.249162    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-cert podName:0cfda7ab-dec3-4780-9dc2-1f220a44ca8d nodeName:}" failed. No retries permitted until 2026-08-03 16:10:41.74914442 +0000 UTC m=+401.298430718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-cert") pod "kserve-controller-manager-d6f4b65b8-wt9m8" (UID: "0cfda7ab-dec3-4780-9dc2-1f220a44ca8d") : secret "kserve-webhook-server-cert" not found
Aug 03 16:10:41.251483 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.251458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f544be1b-e7ba-4bdf-a53e-e6323944b47f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzh7x\" (UID: \"f544be1b-e7ba-4bdf-a53e-e6323944b47f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.261068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.261044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szqh9\" (UniqueName: \"kubernetes.io/projected/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-kube-api-access-szqh9\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.263745 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.263721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2rf\" (UniqueName: \"kubernetes.io/projected/f544be1b-e7ba-4bdf-a53e-e6323944b47f-kube-api-access-qf2rf\") pod \"llmisvc-controller-manager-777c6bd7d5-xzh7x\" (UID: \"f544be1b-e7ba-4bdf-a53e-e6323944b47f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.370540 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.370499    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:41.416986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.416938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:41.560053 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:10:41.560025    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf544be1b_e7ba_4bdf_a53e_e6323944b47f.slice/crio-9e494a214d5dde31f3f8b2cbc63b1ce20366e257d74b449e7a088b7b4b5c4f8d WatchSource:0}: Error finding container 9e494a214d5dde31f3f8b2cbc63b1ce20366e257d74b449e7a088b7b4b5c4f8d: Status 404 returned error can't find the container with id 9e494a214d5dde31f3f8b2cbc63b1ce20366e257d74b449e7a088b7b4b5c4f8d
Aug 03 16:10:41.564085 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.564055    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"]
Aug 03 16:10:41.565373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.565354    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"]
Aug 03 16:10:41.569734 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.569717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.584608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.584588    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 03 16:10:41.592438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.592418    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 03 16:10:41.607953 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.607929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:41.620471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.620450    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"]
Aug 03 16:10:41.652378 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.652347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gthg\" (UniqueName: \"kubernetes.io/projected/d3a53a64-844f-438c-8759-cebeb5763647-kube-api-access-6gthg\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.652511 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.652401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a53a64-844f-438c-8759-cebeb5763647-tls-certs\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.652511 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.652416    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3a53a64-844f-438c-8759-cebeb5763647-tmp\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.722996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.722950    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:41.753151 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.753123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gthg\" (UniqueName: \"kubernetes.io/projected/d3a53a64-844f-438c-8759-cebeb5763647-kube-api-access-6gthg\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.753235 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.753181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.753235 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.753213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a53a64-844f-438c-8759-cebeb5763647-tls-certs\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.753317 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.753235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3a53a64-844f-438c-8759-cebeb5763647-tmp\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.755593 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.755563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3a53a64-844f-438c-8759-cebeb5763647-tmp\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.755719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.755699    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a53a64-844f-438c-8759-cebeb5763647-tls-certs\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.755763 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.755708    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cfda7ab-dec3-4780-9dc2-1f220a44ca8d-cert\") pod \"kserve-controller-manager-d6f4b65b8-wt9m8\" (UID: \"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:41.763340 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.763323    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gthg\" (UniqueName: \"kubernetes.io/projected/d3a53a64-844f-438c-8759-cebeb5763647-kube-api-access-6gthg\") pod \"kube-auth-proxy-845c84c75b-ttdw7\" (UID: \"d3a53a64-844f-438c-8759-cebeb5763647\") " pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.838284 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.838203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x" event={"ID":"f544be1b-e7ba-4bdf-a53e-e6323944b47f","Type":"ContainerStarted","Data":"9e494a214d5dde31f3f8b2cbc63b1ce20366e257d74b449e7a088b7b4b5c4f8d"}
Aug 03 16:10:41.880537 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.880508    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"
Aug 03 16:10:41.959854 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:41.959821    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:42.011167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:42.011063    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7"]
Aug 03 16:10:42.014516 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:10:42.014473    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a53a64_844f_438c_8759_cebeb5763647.slice/crio-3841ec73ccbe398dec61dcc954ce35db970e4ce020657ca7cef2b021e308d1a8 WatchSource:0}: Error finding container 3841ec73ccbe398dec61dcc954ce35db970e4ce020657ca7cef2b021e308d1a8: Status 404 returned error can't find the container with id 3841ec73ccbe398dec61dcc954ce35db970e4ce020657ca7cef2b021e308d1a8
Aug 03 16:10:42.103190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:42.103169    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"]
Aug 03 16:10:42.105131 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:10:42.105110    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfda7ab_dec3_4780_9dc2_1f220a44ca8d.slice/crio-8af8e0bb6d43b40e3b3ffcccf77496c2ef2051fb9e9ac097911e2b3b0c1118b5 WatchSource:0}: Error finding container 8af8e0bb6d43b40e3b3ffcccf77496c2ef2051fb9e9ac097911e2b3b0c1118b5: Status 404 returned error can't find the container with id 8af8e0bb6d43b40e3b3ffcccf77496c2ef2051fb9e9ac097911e2b3b0c1118b5
Aug 03 16:10:42.322548 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:42.322490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:42.417099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:42.417016    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:42.845964 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:42.845926    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8" event={"ID":"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d","Type":"ContainerStarted","Data":"8af8e0bb6d43b40e3b3ffcccf77496c2ef2051fb9e9ac097911e2b3b0c1118b5"}
Aug 03 16:10:42.847368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:42.847338    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7" event={"ID":"d3a53a64-844f-438c-8759-cebeb5763647","Type":"ContainerStarted","Data":"3841ec73ccbe398dec61dcc954ce35db970e4ce020657ca7cef2b021e308d1a8"}
Aug 03 16:10:43.695592 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.695420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:43.717108 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.716909    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"]
Aug 03 16:10:43.721724 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.721685    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.728120 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.728095    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 03 16:10:43.728343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.728323    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 03 16:10:43.728515 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.728498    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 03 16:10:43.728745 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.728729    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w7kv5\""
Aug 03 16:10:43.728990 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.728973    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 03 16:10:43.734047 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.734024    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"]
Aug 03 16:10:43.809005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.808838    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:43.872240 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps62z\" (UniqueName: \"kubernetes.io/projected/28fb93ca-cb71-43b9-ba03-bee7f777a270-kube-api-access-ps62z\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.872436 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872269    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fb93ca-cb71-43b9-ba03-bee7f777a270-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.872436 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.872436 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872380    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.872436 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872427    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.872700 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872450    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.872700 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.872476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.973482 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.972921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.973769    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.973810    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.973843    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.973881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps62z\" (UniqueName: \"kubernetes.io/projected/28fb93ca-cb71-43b9-ba03-bee7f777a270-kube-api-access-ps62z\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.973933    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fb93ca-cb71-43b9-ba03-bee7f777a270-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.973976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.975459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.975243    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.978516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.978473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.979387 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.979344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.981662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.981638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fb93ca-cb71-43b9-ba03-bee7f777a270-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.984063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.984020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.987605 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.987565    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps62z\" (UniqueName: \"kubernetes.io/projected/28fb93ca-cb71-43b9-ba03-bee7f777a270-kube-api-access-ps62z\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:43.989729 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:43.989691    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fb93ca-cb71-43b9-ba03-bee7f777a270-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-2r545\" (UID: \"28fb93ca-cb71-43b9-ba03-bee7f777a270\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:44.050553 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:44.050499    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:44.245715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:44.245278    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"]
Aug 03 16:10:45.123355 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:45.123318    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:45.490900 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:10:45.490829    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fb93ca_cb71_43b9_ba03_bee7f777a270.slice/crio-285923eb017c61d6808fd3f8985bb7515e19659a5ba0a461b62dfa826b869eb2 WatchSource:0}: Error finding container 285923eb017c61d6808fd3f8985bb7515e19659a5ba0a461b62dfa826b869eb2: Status 404 returned error can't find the container with id 285923eb017c61d6808fd3f8985bb7515e19659a5ba0a461b62dfa826b869eb2
Aug 03 16:10:45.868842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:45.868800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545" event={"ID":"28fb93ca-cb71-43b9-ba03-bee7f777a270","Type":"ContainerStarted","Data":"285923eb017c61d6808fd3f8985bb7515e19659a5ba0a461b62dfa826b869eb2"}
Aug 03 16:10:45.950258 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:45.950214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:46.051809 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.051776    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:46.877002 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.876960    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x" event={"ID":"f544be1b-e7ba-4bdf-a53e-e6323944b47f","Type":"ContainerStarted","Data":"fb547d2b55f3fa371679a7996e05c78d25fe6fc42117e930da1287a8fa086763"}
Aug 03 16:10:46.877480 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.877050    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:10:46.879269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.879125    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8" event={"ID":"0cfda7ab-dec3-4780-9dc2-1f220a44ca8d","Type":"ContainerStarted","Data":"3b723c0b8e18511b2e06c6aca6828f72496eb312d175c3f735568201a25a178d"}
Aug 03 16:10:46.879269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.879192    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:10:46.881156 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.881103    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7" event={"ID":"d3a53a64-844f-438c-8759-cebeb5763647","Type":"ContainerStarted","Data":"b4bb35dd1a177758c474291036a6e1c3e3ffb002ae2f872af46beac3f708ee60"}
Aug 03 16:10:46.894885 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.894826    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x" podStartSLOduration=0.730118627 podStartE2EDuration="5.894805657s" totalImagesPullingTime="5.16468703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:46.893281657 +0000 UTC m=+406.442567985" watchObservedRunningTime="2026-08-03 16:10:46.894805657 +0000 UTC m=+406.444091965"
Aug 03 16:10:46.914543 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.914144    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8" podStartSLOduration=1.2884833869999999 podStartE2EDuration="5.914116626s" totalImagesPullingTime="4.625633239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:46.9132638 +0000 UTC m=+406.462550116" watchObservedRunningTime="2026-08-03 16:10:46.914116626 +0000 UTC m=+406.463402944"
Aug 03 16:10:46.935373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:46.934035    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-845c84c75b-ttdw7" podStartSLOduration=1.212363535 podStartE2EDuration="5.934017711s" totalImagesPullingTime="4.721654176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:46.930235194 +0000 UTC m=+406.479521504" watchObservedRunningTime="2026-08-03 16:10:46.934017711 +0000 UTC m=+406.483304028"
Aug 03 16:10:48.555279 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:48.555242    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 03 16:10:48.555542 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:48.555312    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 03 16:10:48.893914 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:48.893807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545" event={"ID":"28fb93ca-cb71-43b9-ba03-bee7f777a270","Type":"ContainerStarted","Data":"eccbc5901d799f1373344eac8d76d15568b9eac0e12adb8a03680a2b69816ea8"}
Aug 03 16:10:48.894090 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:48.893943    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:10:48.918486 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:48.918435    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545" podStartSLOduration=2.856456302 podStartE2EDuration="5.918418279s" totalImagesPullingTime="3.061961977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:48.917004121 +0000 UTC m=+408.466290452" watchObservedRunningTime="2026-08-03 16:10:48.918418279 +0000 UTC m=+408.467704595"
Aug 03 16:10:49.584156 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:49.584107    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:49.653982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:49.653936    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:49.888364 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:49.888270    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:49.901105 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:49.901072    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-2r545 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 03 16:10:49.901261 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:49.901130    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545" podUID="28fb93ca-cb71-43b9-ba03-bee7f777a270" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 03 16:10:50.043852 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:50.043815    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:50.289754 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:50.289727    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:10:52.900304 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:10:52.900272    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-2r545"
Aug 03 16:11:17.889106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:17.889072    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzh7x"
Aug 03 16:11:17.892059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:17.892032    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wt9m8"
Aug 03 16:11:32.290246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:32.290212    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:32.387951 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:32.387919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:44.245854 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:44.245817    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:45.042221 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:45.042186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:45.846031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:45.845997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:49.117102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:49.117055    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:49.261818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:49.261785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:49.366554 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:49.366503    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:51.264918 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:51.264883    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:51.507212 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:51.507182    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:52.924444 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:52.924415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:53.055780 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:53.055746    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:53.154177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:53.154144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:53.670071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:53.670035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:55.076342 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:55.076311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:55.978679 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:55.973904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:56.643982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:56.643952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:59.456034 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:59.456000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:59.526046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:59.526010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:59.849368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:59.849334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:11:59.950773 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:11:59.950742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:00.265466 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:00.265379    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:00.330871 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:00.330836    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:07.112203 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:07.112159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:07.307846 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:07.307816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:08.528423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:08.528380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:08.628101 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:08.628070    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:11.651491 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:11.651457    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:11.753565 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:11.753513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:15.466256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:15.466218    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:15.566291 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:15.566259    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:20.682243 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:20.682205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:20.788000 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:20.787955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:21.073384 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:21.073356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:21.174135 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:21.174095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:21.284841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:21.284812    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:21.382896 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:21.382809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:27.732414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:27.732378    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:27.795958 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:27.795929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:32.484735 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:32.484700    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:32.587897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:32.587857    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:34.791717 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:34.791685    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:34.979009 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:34.978978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:37.287043 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:37.287010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:37.389177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:37.389140    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:46.711582 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.711522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:46.712273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.712241    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:12:46.715697 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.715679    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:46.718345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.718323    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 03 16:12:46.718438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.718368    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 03 16:12:46.719424 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.719392    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-psr2n\""
Aug 03 16:12:46.719424 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.719406    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 03 16:12:46.723907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.723889    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:12:46.808632 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.808603    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:12:46.819733 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.819706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a37eec17-590e-4f14-b10a-291178b226c9-config-file\") pod \"limitador-limitador-859c4d96d6-gk8q8\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:46.819907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.819752    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z97g\" (UniqueName: \"kubernetes.io/projected/a37eec17-590e-4f14-b10a-291178b226c9-kube-api-access-7z97g\") pod \"limitador-limitador-859c4d96d6-gk8q8\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:46.835571 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.835515    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:46.893499 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.893471    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:46.920290 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.920257    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a37eec17-590e-4f14-b10a-291178b226c9-config-file\") pod \"limitador-limitador-859c4d96d6-gk8q8\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:46.920426 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.920314    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z97g\" (UniqueName: \"kubernetes.io/projected/a37eec17-590e-4f14-b10a-291178b226c9-kube-api-access-7z97g\") pod \"limitador-limitador-859c4d96d6-gk8q8\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:46.920880 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.920861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a37eec17-590e-4f14-b10a-291178b226c9-config-file\") pod \"limitador-limitador-859c4d96d6-gk8q8\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:46.928219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:46.928189    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z97g\" (UniqueName: \"kubernetes.io/projected/a37eec17-590e-4f14-b10a-291178b226c9-kube-api-access-7z97g\") pod \"limitador-limitador-859c4d96d6-gk8q8\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:47.026359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:47.026295    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:47.158262 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:47.158239    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:12:47.160296 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:12:47.160267    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37eec17_590e_4f14_b10a_291178b226c9.slice/crio-c6514bcd1ddc6e1027fc7e99eebae6c336605db7a18c8038caa00fad36935520 WatchSource:0}: Error finding container c6514bcd1ddc6e1027fc7e99eebae6c336605db7a18c8038caa00fad36935520: Status 404 returned error can't find the container with id c6514bcd1ddc6e1027fc7e99eebae6c336605db7a18c8038caa00fad36935520
Aug 03 16:12:47.411923 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:47.411892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8" event={"ID":"a37eec17-590e-4f14-b10a-291178b226c9","Type":"ContainerStarted","Data":"c6514bcd1ddc6e1027fc7e99eebae6c336605db7a18c8038caa00fad36935520"}
Aug 03 16:12:47.562042 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:47.562002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:47.659801 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:47.659770    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:12:50.429660 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:50.429624    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8" event={"ID":"a37eec17-590e-4f14-b10a-291178b226c9","Type":"ContainerStarted","Data":"1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f"}
Aug 03 16:12:50.430053 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:50.429692    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:12:50.447185 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:12:50.447130    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8" podStartSLOduration=2.083434786 podStartE2EDuration="4.447096847s" totalImagesPullingTime="2.363662061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:50.445606424 +0000 UTC m=+529.994892741" watchObservedRunningTime="2026-08-03 16:12:50.447096847 +0000 UTC m=+529.996383164"
Aug 03 16:13:00.322070 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.321980    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:13:00.322614 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.322243    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8" podUID="a37eec17-590e-4f14-b10a-291178b226c9" containerName="limitador" containerID="cri-o://1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f" gracePeriod=30
Aug 03 16:13:00.322990 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.322942    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:13:00.865107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.865083    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:13:00.933896 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.933813    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a37eec17-590e-4f14-b10a-291178b226c9-config-file\") pod \"a37eec17-590e-4f14-b10a-291178b226c9\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") "
Aug 03 16:13:00.934037 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.933989    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z97g\" (UniqueName: \"kubernetes.io/projected/a37eec17-590e-4f14-b10a-291178b226c9-kube-api-access-7z97g\") pod \"a37eec17-590e-4f14-b10a-291178b226c9\" (UID: \"a37eec17-590e-4f14-b10a-291178b226c9\") "
Aug 03 16:13:00.934173 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.934148    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37eec17-590e-4f14-b10a-291178b226c9-config-file" (OuterVolumeSpecName: "config-file") pod "a37eec17-590e-4f14-b10a-291178b226c9" (UID: "a37eec17-590e-4f14-b10a-291178b226c9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:13:00.936157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:00.936133    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37eec17-590e-4f14-b10a-291178b226c9-kube-api-access-7z97g" (OuterVolumeSpecName: "kube-api-access-7z97g") pod "a37eec17-590e-4f14-b10a-291178b226c9" (UID: "a37eec17-590e-4f14-b10a-291178b226c9"). InnerVolumeSpecName "kube-api-access-7z97g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:13:01.035434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.035403    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z97g\" (UniqueName: \"kubernetes.io/projected/a37eec17-590e-4f14-b10a-291178b226c9-kube-api-access-7z97g\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:13:01.035434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.035431    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a37eec17-590e-4f14-b10a-291178b226c9-config-file\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:13:01.484036 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.484001    2560 generic.go:356] "Generic (PLEG): container finished" podID="a37eec17-590e-4f14-b10a-291178b226c9" containerID="1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f" exitCode=0
Aug 03 16:13:01.484448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.484040    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"
Aug 03 16:13:01.484448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.484072    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8" event={"ID":"a37eec17-590e-4f14-b10a-291178b226c9","Type":"ContainerDied","Data":"1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f"}
Aug 03 16:13:01.484448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.484111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-gk8q8" event={"ID":"a37eec17-590e-4f14-b10a-291178b226c9","Type":"ContainerDied","Data":"c6514bcd1ddc6e1027fc7e99eebae6c336605db7a18c8038caa00fad36935520"}
Aug 03 16:13:01.484448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.484132    2560 scope.go:117] "RemoveContainer" containerID="1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f"
Aug 03 16:13:01.492378 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.492360    2560 scope.go:117] "RemoveContainer" containerID="1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f"
Aug 03 16:13:01.492670 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:13:01.492649    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f\": container with ID starting with 1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f not found: ID does not exist" containerID="1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f"
Aug 03 16:13:01.492734 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.492679    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f"} err="failed to get container status \"1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f\": rpc error: code = NotFound desc = could not find container \"1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f\": container with ID starting with 1d293a8cd6708dfeaf2b9e0cded31d66febba14c4b927dc78855e10d58b2990f not found: ID does not exist"
Aug 03 16:13:01.502302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.502281    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:13:01.506152 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:01.506131    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-gk8q8"]
Aug 03 16:13:02.941995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:02.941954    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:02.986344 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:02.986310    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a37eec17-590e-4f14-b10a-291178b226c9" path="/var/lib/kubelet/pods/a37eec17-590e-4f14-b10a-291178b226c9/volumes"
Aug 03 16:13:03.042052 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.042013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:03.279681 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.279603    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"]
Aug 03 16:13:03.279965 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.279952    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a37eec17-590e-4f14-b10a-291178b226c9" containerName="limitador"
Aug 03 16:13:03.280017 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.279967    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a37eec17-590e-4f14-b10a-291178b226c9" containerName="limitador"
Aug 03 16:13:03.280051 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.280022    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a37eec17-590e-4f14-b10a-291178b226c9" containerName="limitador"
Aug 03 16:13:03.284384 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.284350    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.287462 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.287439    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 03 16:13:03.287621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.287480    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 03 16:13:03.288537 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.288506    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v5fc4\""
Aug 03 16:13:03.288667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.288511    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 03 16:13:03.292265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.292242    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"]
Aug 03 16:13:03.327562 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.327511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:03.354638 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.354602    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2cr\" (UniqueName: \"kubernetes.io/projected/6a69665a-9486-4ac5-bcfb-49c8abe964e3-kube-api-access-hk2cr\") pod \"postgres-69fb4594cf-dwv92\" (UID: \"6a69665a-9486-4ac5-bcfb-49c8abe964e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.354788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.354661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6a69665a-9486-4ac5-bcfb-49c8abe964e3-data\") pod \"postgres-69fb4594cf-dwv92\" (UID: \"6a69665a-9486-4ac5-bcfb-49c8abe964e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.455723 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.455680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2cr\" (UniqueName: \"kubernetes.io/projected/6a69665a-9486-4ac5-bcfb-49c8abe964e3-kube-api-access-hk2cr\") pod \"postgres-69fb4594cf-dwv92\" (UID: \"6a69665a-9486-4ac5-bcfb-49c8abe964e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.455917 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.455733    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6a69665a-9486-4ac5-bcfb-49c8abe964e3-data\") pod \"postgres-69fb4594cf-dwv92\" (UID: \"6a69665a-9486-4ac5-bcfb-49c8abe964e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.456144 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.456123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6a69665a-9486-4ac5-bcfb-49c8abe964e3-data\") pod \"postgres-69fb4594cf-dwv92\" (UID: \"6a69665a-9486-4ac5-bcfb-49c8abe964e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.465117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.465088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2cr\" (UniqueName: \"kubernetes.io/projected/6a69665a-9486-4ac5-bcfb-49c8abe964e3-kube-api-access-hk2cr\") pod \"postgres-69fb4594cf-dwv92\" (UID: \"6a69665a-9486-4ac5-bcfb-49c8abe964e3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.594918 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.594882    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:03.727155 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:03.727129    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"]
Aug 03 16:13:03.729160 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:13:03.729137    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a69665a_9486_4ac5_bcfb_49c8abe964e3.slice/crio-4e8fd416d29e60af2ffaa851ae144bd3b47d6ca04d8107039c64d5f4b58c8228 WatchSource:0}: Error finding container 4e8fd416d29e60af2ffaa851ae144bd3b47d6ca04d8107039c64d5f4b58c8228: Status 404 returned error can't find the container with id 4e8fd416d29e60af2ffaa851ae144bd3b47d6ca04d8107039c64d5f4b58c8228
Aug 03 16:13:04.506853 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:04.506809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92" event={"ID":"6a69665a-9486-4ac5-bcfb-49c8abe964e3","Type":"ContainerStarted","Data":"4e8fd416d29e60af2ffaa851ae144bd3b47d6ca04d8107039c64d5f4b58c8228"}
Aug 03 16:13:08.529468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:08.529432    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92" event={"ID":"6a69665a-9486-4ac5-bcfb-49c8abe964e3","Type":"ContainerStarted","Data":"04173d9b5ce0d621ae184b792d988e940c076f85e1e66ca876b846d860894a6c"}
Aug 03 16:13:08.529856 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:08.529548    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:08.545720 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:08.545675    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92" podStartSLOduration=0.860970558 podStartE2EDuration="5.545661593s" totalImagesPullingTime="4.684691035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:08.545163388 +0000 UTC m=+548.094449706" watchObservedRunningTime="2026-08-03 16:13:08.545661593 +0000 UTC m=+548.094947909"
Aug 03 16:13:14.563499 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:14.563467    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dwv92"
Aug 03 16:13:14.907343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:14.907263    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:15.391510 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:15.391461    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:22.367397 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.367360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:22.367857 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.367721    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579f596d7b-ldw8h"]
Aug 03 16:13:22.397602 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.397574    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579f596d7b-ldw8h"]
Aug 03 16:13:22.397762 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.397666    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.400898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.400874    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 03 16:13:22.401024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.400897    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 03 16:13:22.401024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.400874    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kmz6t\""
Aug 03 16:13:22.424067 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.424022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:22.483577 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.483519    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:22.521694 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.521664    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4vvl\" (UniqueName: \"kubernetes.io/projected/640673ef-c599-463d-85a6-d6d51b8b8ad9-kube-api-access-d4vvl\") pod \"maas-controller-579f596d7b-ldw8h\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") " pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.521858 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.521708    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640673ef-c599-463d-85a6-d6d51b8b8ad9-webhook-cert\") pod \"maas-controller-579f596d7b-ldw8h\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") " pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.540096 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.540065    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69978859c5-h5jqd"]
Aug 03 16:13:22.543463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.543443    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.552072 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.552045    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69978859c5-h5jqd"]
Aug 03 16:13:22.598210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.598175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:22.622579 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.622465    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4vvl\" (UniqueName: \"kubernetes.io/projected/640673ef-c599-463d-85a6-d6d51b8b8ad9-kube-api-access-d4vvl\") pod \"maas-controller-579f596d7b-ldw8h\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") " pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.622579 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.622504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mppxc\" (UniqueName: \"kubernetes.io/projected/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-kube-api-access-mppxc\") pod \"maas-controller-69978859c5-h5jqd\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") " pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.622579 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.622548    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-webhook-cert\") pod \"maas-controller-69978859c5-h5jqd\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") " pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.622857 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.622609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640673ef-c599-463d-85a6-d6d51b8b8ad9-webhook-cert\") pod \"maas-controller-579f596d7b-ldw8h\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") " pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.625030 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.625000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640673ef-c599-463d-85a6-d6d51b8b8ad9-webhook-cert\") pod \"maas-controller-579f596d7b-ldw8h\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") " pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.631086 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.631064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4vvl\" (UniqueName: \"kubernetes.io/projected/640673ef-c599-463d-85a6-d6d51b8b8ad9-kube-api-access-d4vvl\") pod \"maas-controller-579f596d7b-ldw8h\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") " pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.708550 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.708499    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:22.723991 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.723963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mppxc\" (UniqueName: \"kubernetes.io/projected/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-kube-api-access-mppxc\") pod \"maas-controller-69978859c5-h5jqd\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") " pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.723991 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.724004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-webhook-cert\") pod \"maas-controller-69978859c5-h5jqd\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") " pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.726494 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.726465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-webhook-cert\") pod \"maas-controller-69978859c5-h5jqd\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") " pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.733353 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.733327    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mppxc\" (UniqueName: \"kubernetes.io/projected/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-kube-api-access-mppxc\") pod \"maas-controller-69978859c5-h5jqd\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") " pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.844647 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.844620    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579f596d7b-ldw8h"]
Aug 03 16:13:22.846499 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:13:22.846476    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640673ef_c599_463d_85a6_d6d51b8b8ad9.slice/crio-bf5540b5cee07a70ac76d646efc3f25a098e6faf14b1472dcad3f0af347aea27 WatchSource:0}: Error finding container bf5540b5cee07a70ac76d646efc3f25a098e6faf14b1472dcad3f0af347aea27: Status 404 returned error can't find the container with id bf5540b5cee07a70ac76d646efc3f25a098e6faf14b1472dcad3f0af347aea27
Aug 03 16:13:22.854064 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.854039    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:22.987843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:22.987814    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69978859c5-h5jqd"]
Aug 03 16:13:22.988092 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:13:22.988070    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda639e974_dd0b_4fe2_9fb5_f83b92e177e7.slice/crio-134f3e0915c4148c4de24431a9622ca8fe9daabb866ff15d106496fe03e866ec WatchSource:0}: Error finding container 134f3e0915c4148c4de24431a9622ca8fe9daabb866ff15d106496fe03e866ec: Status 404 returned error can't find the container with id 134f3e0915c4148c4de24431a9622ca8fe9daabb866ff15d106496fe03e866ec
Aug 03 16:13:23.598014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:23.597983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579f596d7b-ldw8h" event={"ID":"640673ef-c599-463d-85a6-d6d51b8b8ad9","Type":"ContainerStarted","Data":"bf5540b5cee07a70ac76d646efc3f25a098e6faf14b1472dcad3f0af347aea27"}
Aug 03 16:13:23.599075 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:23.599054    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-h5jqd" event={"ID":"a639e974-dd0b-4fe2-9fb5-f83b92e177e7","Type":"ContainerStarted","Data":"134f3e0915c4148c4de24431a9622ca8fe9daabb866ff15d106496fe03e866ec"}
Aug 03 16:13:24.607167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.606600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579f596d7b-ldw8h" event={"ID":"640673ef-c599-463d-85a6-d6d51b8b8ad9","Type":"ContainerStarted","Data":"ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168"}
Aug 03 16:13:24.607167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.606709    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:24.608326 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.608300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-h5jqd" event={"ID":"a639e974-dd0b-4fe2-9fb5-f83b92e177e7","Type":"ContainerStarted","Data":"afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c"}
Aug 03 16:13:24.608472 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.608383    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:24.629141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.629115    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:24.647975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.647927    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-579f596d7b-ldw8h" podStartSLOduration=1.046491845 podStartE2EDuration="2.647910794s" totalImagesPullingTime="1.601418949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:24.644980179 +0000 UTC m=+564.194266496" watchObservedRunningTime="2026-08-03 16:13:24.647910794 +0000 UTC m=+564.197197110"
Aug 03 16:13:24.669459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.669412    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69978859c5-h5jqd" podStartSLOduration=1.206551299 podStartE2EDuration="2.669396653s" totalImagesPullingTime="1.462845354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:24.668008701 +0000 UTC m=+564.217295027" watchObservedRunningTime="2026-08-03 16:13:24.669396653 +0000 UTC m=+564.218682968"
Aug 03 16:13:24.731298 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:24.731195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:35.616080 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.616052    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:13:35.616576 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.616126    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:35.692368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.692332    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579f596d7b-ldw8h"]
Aug 03 16:13:35.692624 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.692599    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-579f596d7b-ldw8h" podUID="640673ef-c599-463d-85a6-d6d51b8b8ad9" containerName="manager" containerID="cri-o://ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168" gracePeriod=30
Aug 03 16:13:35.939205 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.939180    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:35.940172 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.940154    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640673ef-c599-463d-85a6-d6d51b8b8ad9-webhook-cert\") pod \"640673ef-c599-463d-85a6-d6d51b8b8ad9\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") "
Aug 03 16:13:35.940273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.940216    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4vvl\" (UniqueName: \"kubernetes.io/projected/640673ef-c599-463d-85a6-d6d51b8b8ad9-kube-api-access-d4vvl\") pod \"640673ef-c599-463d-85a6-d6d51b8b8ad9\" (UID: \"640673ef-c599-463d-85a6-d6d51b8b8ad9\") "
Aug 03 16:13:35.942333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.942301    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/640673ef-c599-463d-85a6-d6d51b8b8ad9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "640673ef-c599-463d-85a6-d6d51b8b8ad9" (UID: "640673ef-c599-463d-85a6-d6d51b8b8ad9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:13:35.942333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:35.942301    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/640673ef-c599-463d-85a6-d6d51b8b8ad9-kube-api-access-d4vvl" (OuterVolumeSpecName: "kube-api-access-d4vvl") pod "640673ef-c599-463d-85a6-d6d51b8b8ad9" (UID: "640673ef-c599-463d-85a6-d6d51b8b8ad9"). InnerVolumeSpecName "kube-api-access-d4vvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:13:36.041369 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.041333    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640673ef-c599-463d-85a6-d6d51b8b8ad9-webhook-cert\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:13:36.041369 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.041362    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4vvl\" (UniqueName: \"kubernetes.io/projected/640673ef-c599-463d-85a6-d6d51b8b8ad9-kube-api-access-d4vvl\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:13:36.669289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.669252    2560 generic.go:356] "Generic (PLEG): container finished" podID="640673ef-c599-463d-85a6-d6d51b8b8ad9" containerID="ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168" exitCode=0
Aug 03 16:13:36.669719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.669299    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579f596d7b-ldw8h"
Aug 03 16:13:36.669719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.669299    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579f596d7b-ldw8h" event={"ID":"640673ef-c599-463d-85a6-d6d51b8b8ad9","Type":"ContainerDied","Data":"ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168"}
Aug 03 16:13:36.669719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.669410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579f596d7b-ldw8h" event={"ID":"640673ef-c599-463d-85a6-d6d51b8b8ad9","Type":"ContainerDied","Data":"bf5540b5cee07a70ac76d646efc3f25a098e6faf14b1472dcad3f0af347aea27"}
Aug 03 16:13:36.669719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.669437    2560 scope.go:117] "RemoveContainer" containerID="ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168"
Aug 03 16:13:36.678081 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.678063    2560 scope.go:117] "RemoveContainer" containerID="ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168"
Aug 03 16:13:36.678325 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:13:36.678306    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168\": container with ID starting with ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168 not found: ID does not exist" containerID="ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168"
Aug 03 16:13:36.678386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.678333    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168"} err="failed to get container status \"ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168\": rpc error: code = NotFound desc = could not find container \"ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168\": container with ID starting with ab243b33d36fd92e68de0cb7ecba93417c9957288fcce558f0bfe6f727409168 not found: ID does not exist"
Aug 03 16:13:36.690231 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.690196    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579f596d7b-ldw8h"]
Aug 03 16:13:36.693476 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.693455    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579f596d7b-ldw8h"]
Aug 03 16:13:36.986988 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:36.986912    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="640673ef-c599-463d-85a6-d6d51b8b8ad9" path="/var/lib/kubelet/pods/640673ef-c599-463d-85a6-d6d51b8b8ad9/volumes"
Aug 03 16:13:38.528641 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:38.528604    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:38.627823 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:38.627781    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:39.274024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:39.273983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:53.111204 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:53.111174    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:53.207183 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:53.207139    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:54.608360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.608323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:54.739786 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.739755    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"]
Aug 03 16:13:54.740182 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.740164    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640673ef-c599-463d-85a6-d6d51b8b8ad9" containerName="manager"
Aug 03 16:13:54.740182 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.740184    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="640673ef-c599-463d-85a6-d6d51b8b8ad9" containerName="manager"
Aug 03 16:13:54.740333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.740303    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="640673ef-c599-463d-85a6-d6d51b8b8ad9" containerName="manager"
Aug 03 16:13:54.744822 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.744796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.748598 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.748574    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 03 16:13:54.748737 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.748697    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jpp8w\""
Aug 03 16:13:54.752892 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.752859    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"]
Aug 03 16:13:54.789012 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.788981    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:54.800980 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.800954    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-maas-api-tls\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.801109 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.800999    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-openshift-service-ca\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.801109 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.801052    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqftd\" (UniqueName: \"kubernetes.io/projected/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-kube-api-access-xqftd\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.902335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.902249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqftd\" (UniqueName: \"kubernetes.io/projected/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-kube-api-access-xqftd\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.902335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.902329    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-maas-api-tls\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.902511 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.902366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-openshift-service-ca\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.903046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.903020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-openshift-service-ca\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.904903 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.904885    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-maas-api-tls\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:54.910865 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:54.910829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqftd\" (UniqueName: \"kubernetes.io/projected/25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0-kube-api-access-xqftd\") pod \"maas-api-6cd6bc484-t26c5\" (UID: \"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0\") " pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:55.058410 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.058370    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:55.138610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.138576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:55.201591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.201566    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"]
Aug 03 16:13:55.203920 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.203876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:13:55.204059 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:13:55.204025    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b6c0dd_e8d2_4e00_a25e_dcb60c1f8db0.slice/crio-8982d1fd2fc216a1ce00c95a6460c00e5bf3f1bdd410eb27723e046dc0fca44d WatchSource:0}: Error finding container 8982d1fd2fc216a1ce00c95a6460c00e5bf3f1bdd410eb27723e046dc0fca44d: Status 404 returned error can't find the container with id 8982d1fd2fc216a1ce00c95a6460c00e5bf3f1bdd410eb27723e046dc0fca44d
Aug 03 16:13:55.245333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.245298    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"]
Aug 03 16:13:55.248824 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.248803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.251291 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.251274    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wz5s8\""
Aug 03 16:13:55.251364 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.251328    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 03 16:13:55.257937 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.257918    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"]
Aug 03 16:13:55.285236 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.285205    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-lstfn"]
Aug 03 16:13:55.288431 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.288403    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.299178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.299157    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-lstfn"]
Aug 03 16:13:55.306690 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.306660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f4e207-00be-4317-aa53-7b27628a9be5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-bd7sz\" (UID: \"f0f4e207-00be-4317-aa53-7b27628a9be5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.306798 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.306694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt28r\" (UniqueName: \"kubernetes.io/projected/f0f4e207-00be-4317-aa53-7b27628a9be5-kube-api-access-zt28r\") pod \"payload-pre-processing-687dbb6cf6-bd7sz\" (UID: \"f0f4e207-00be-4317-aa53-7b27628a9be5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.408083 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.408046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f4e207-00be-4317-aa53-7b27628a9be5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-bd7sz\" (UID: \"f0f4e207-00be-4317-aa53-7b27628a9be5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.408083 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.408084    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt28r\" (UniqueName: \"kubernetes.io/projected/f0f4e207-00be-4317-aa53-7b27628a9be5-kube-api-access-zt28r\") pod \"payload-pre-processing-687dbb6cf6-bd7sz\" (UID: \"f0f4e207-00be-4317-aa53-7b27628a9be5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.408331 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.408120    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x8sw\" (UniqueName: \"kubernetes.io/projected/8e84fafc-9d3d-4122-9b7c-67e622ad5028-kube-api-access-9x8sw\") pod \"payload-processing-5cc6cd444b-lstfn\" (UID: \"8e84fafc-9d3d-4122-9b7c-67e622ad5028\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.408331 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.408162    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e84fafc-9d3d-4122-9b7c-67e622ad5028-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-lstfn\" (UID: \"8e84fafc-9d3d-4122-9b7c-67e622ad5028\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.408806 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.408787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f4e207-00be-4317-aa53-7b27628a9be5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-bd7sz\" (UID: \"f0f4e207-00be-4317-aa53-7b27628a9be5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.418658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.418630    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt28r\" (UniqueName: \"kubernetes.io/projected/f0f4e207-00be-4317-aa53-7b27628a9be5-kube-api-access-zt28r\") pod \"payload-pre-processing-687dbb6cf6-bd7sz\" (UID: \"f0f4e207-00be-4317-aa53-7b27628a9be5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.509169 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.509072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x8sw\" (UniqueName: \"kubernetes.io/projected/8e84fafc-9d3d-4122-9b7c-67e622ad5028-kube-api-access-9x8sw\") pod \"payload-processing-5cc6cd444b-lstfn\" (UID: \"8e84fafc-9d3d-4122-9b7c-67e622ad5028\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.509169 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.509132    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e84fafc-9d3d-4122-9b7c-67e622ad5028-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-lstfn\" (UID: \"8e84fafc-9d3d-4122-9b7c-67e622ad5028\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.509855 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.509833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e84fafc-9d3d-4122-9b7c-67e622ad5028-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-lstfn\" (UID: \"8e84fafc-9d3d-4122-9b7c-67e622ad5028\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.520993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.520970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x8sw\" (UniqueName: \"kubernetes.io/projected/8e84fafc-9d3d-4122-9b7c-67e622ad5028-kube-api-access-9x8sw\") pod \"payload-processing-5cc6cd444b-lstfn\" (UID: \"8e84fafc-9d3d-4122-9b7c-67e622ad5028\") " pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.559848 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.559812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:55.599648 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.599616    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:55.705980 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.705947    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"]
Aug 03 16:13:55.709822 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:13:55.709788    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f4e207_00be_4317_aa53_7b27628a9be5.slice/crio-06c9c7db85736ff468edbe01e8d209929beed3f5af03473d60a5c45965d98517 WatchSource:0}: Error finding container 06c9c7db85736ff468edbe01e8d209929beed3f5af03473d60a5c45965d98517: Status 404 returned error can't find the container with id 06c9c7db85736ff468edbe01e8d209929beed3f5af03473d60a5c45965d98517
Aug 03 16:13:55.752911 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.752884    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-lstfn"]
Aug 03 16:13:55.754879 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:13:55.754851    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e84fafc_9d3d_4122_9b7c_67e622ad5028.slice/crio-5c2c2f4db876aee81884b28d378e65281bca44747ac20c66a47a0f01658f5569 WatchSource:0}: Error finding container 5c2c2f4db876aee81884b28d378e65281bca44747ac20c66a47a0f01658f5569: Status 404 returned error can't find the container with id 5c2c2f4db876aee81884b28d378e65281bca44747ac20c66a47a0f01658f5569
Aug 03 16:13:55.756583 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.756469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5" event={"ID":"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0","Type":"ContainerStarted","Data":"8982d1fd2fc216a1ce00c95a6460c00e5bf3f1bdd410eb27723e046dc0fca44d"}
Aug 03 16:13:55.757690 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:55.757666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz" event={"ID":"f0f4e207-00be-4317-aa53-7b27628a9be5","Type":"ContainerStarted","Data":"06c9c7db85736ff468edbe01e8d209929beed3f5af03473d60a5c45965d98517"}
Aug 03 16:13:56.769089 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:56.768988    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn" event={"ID":"8e84fafc-9d3d-4122-9b7c-67e622ad5028","Type":"ContainerStarted","Data":"5c2c2f4db876aee81884b28d378e65281bca44747ac20c66a47a0f01658f5569"}
Aug 03 16:13:57.775560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:57.775457    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5" event={"ID":"25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0","Type":"ContainerStarted","Data":"1dc21cdd64a300f5b8c1eebe83872eaf4ecf2fb7e57be3147d9464baf4efcce2"}
Aug 03 16:13:57.776004 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:57.775592    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:13:57.795644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:57.795590    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5" podStartSLOduration=1.679392617 podStartE2EDuration="3.795573304s" totalImagesPullingTime="2.116180687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:57.793835017 +0000 UTC m=+597.343121338" watchObservedRunningTime="2026-08-03 16:13:57.795573304 +0000 UTC m=+597.344859619"
Aug 03 16:13:58.781810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:58.781714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn" event={"ID":"8e84fafc-9d3d-4122-9b7c-67e622ad5028","Type":"ContainerStarted","Data":"853ffe1869ade89034b62cafa3b9c44ecb226f8182648d631cea3c07e4b85b3c"}
Aug 03 16:13:58.781810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:58.781783    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:13:58.783191 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:58.783156    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz" event={"ID":"f0f4e207-00be-4317-aa53-7b27628a9be5","Type":"ContainerStarted","Data":"3705c7d66572d9d3f92bd4ef6c5c1151152b673896ae57022f70d7a5b1c0c460"}
Aug 03 16:13:58.783366 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:58.783346    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:13:58.800903 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:58.800856    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn" podStartSLOduration=1.111815673 podStartE2EDuration="3.80084377s" totalImagesPullingTime="2.689028097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:58.798568318 +0000 UTC m=+598.347854639" watchObservedRunningTime="2026-08-03 16:13:58.80084377 +0000 UTC m=+598.350130086"
Aug 03 16:13:58.815890 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:13:58.815848    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz" podStartSLOduration=1.087958696 podStartE2EDuration="3.815834972s" totalImagesPullingTime="2.727876276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:58.813836864 +0000 UTC m=+598.363123181" watchObservedRunningTime="2026-08-03 16:13:58.815834972 +0000 UTC m=+598.365121287"
Aug 03 16:14:00.909624 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:00.909593    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:14:00.910078 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:00.909770    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:14:03.788592 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:03.788562    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6cd6bc484-t26c5"
Aug 03 16:14:04.947663 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:04.947631    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:05.072285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:05.072247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:09.791126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:09.791093    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bd7sz"
Aug 03 16:14:09.791601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:09.791269    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-lstfn"
Aug 03 16:14:10.247964 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.247895    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"]
Aug 03 16:14:10.251632 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.251610    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.254343 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.254324    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 03 16:14:10.254446 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.254331    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 03 16:14:10.255416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.255390    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6svrh\""
Aug 03 16:14:10.255487 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.255395    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Aug 03 16:14:10.265948 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.265926    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"]
Aug 03 16:14:10.273076 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.273042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:10.348302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.348268    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.348302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.348306    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/172d98ac-835e-474b-a43a-455aa335c675-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.348519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.348432    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx4bk\" (UniqueName: \"kubernetes.io/projected/172d98ac-835e-474b-a43a-455aa335c675-kube-api-access-gx4bk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.348519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.348491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.348632 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.348566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.348632 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.348584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.449910 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.449877    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450100 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.449928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450100 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.449946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450100 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.450062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450100 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.450099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/172d98ac-835e-474b-a43a-455aa335c675-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450334 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.450183    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx4bk\" (UniqueName: \"kubernetes.io/projected/172d98ac-835e-474b-a43a-455aa335c675-kube-api-access-gx4bk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450476 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.450440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.450842    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.450990 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.450930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.452246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.452229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/172d98ac-835e-474b-a43a-455aa335c675-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.452694 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.452675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/172d98ac-835e-474b-a43a-455aa335c675-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.458750 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.458730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx4bk\" (UniqueName: \"kubernetes.io/projected/172d98ac-835e-474b-a43a-455aa335c675-kube-api-access-gx4bk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5\" (UID: \"172d98ac-835e-474b-a43a-455aa335c675\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.562778 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.562748    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:10.700219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.700191    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"]
Aug 03 16:14:10.702175 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:14:10.702144    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172d98ac_835e_474b_a43a_455aa335c675.slice/crio-5404649f13698af3f46499b0cca260ee42089304bb11323730c3b0b31a5c23e6 WatchSource:0}: Error finding container 5404649f13698af3f46499b0cca260ee42089304bb11323730c3b0b31a5c23e6: Status 404 returned error can't find the container with id 5404649f13698af3f46499b0cca260ee42089304bb11323730c3b0b31a5c23e6
Aug 03 16:14:10.843388 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:10.843305    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5" event={"ID":"172d98ac-835e-474b-a43a-455aa335c675","Type":"ContainerStarted","Data":"5404649f13698af3f46499b0cca260ee42089304bb11323730c3b0b31a5c23e6"}
Aug 03 16:14:12.855351 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:12.855311    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5" event={"ID":"172d98ac-835e-474b-a43a-455aa335c675","Type":"ContainerStarted","Data":"4a64025c4b2760e8774a5835a0038d50a9e3241230eeb2009702099251f852a5"}
Aug 03 16:14:12.873649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:12.873571    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5" podStartSLOduration=1.757454064 podStartE2EDuration="2.873555892s" totalImagesPullingTime="1.116101828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:14:12.872916269 +0000 UTC m=+612.422202608" watchObservedRunningTime="2026-08-03 16:14:12.873555892 +0000 UTC m=+612.422842208"
Aug 03 16:14:19.577192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:19.577159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:20.563953 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:20.563902    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:20.564159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:20.563977    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:20.577323 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:20.577299    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:20.906546 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:20.906442    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5"
Aug 03 16:14:22.332322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:22.332282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:29.144320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.144279    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"]
Aug 03 16:14:29.147862 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.147831    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.150769 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.150745    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Aug 03 16:14:29.160345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.160320    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"]
Aug 03 16:14:29.182739 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.182710    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:29.207923 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.207893    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.207923 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.207923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.208106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.207967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76dj\" (UniqueName: \"kubernetes.io/projected/92d1c354-a475-41fc-832e-795cdff274d4-kube-api-access-v76dj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.208106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.207998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.208106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.208035    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92d1c354-a475-41fc-832e-795cdff274d4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.208106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.208062    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92d1c354-a475-41fc-832e-795cdff274d4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309073    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309311 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309311 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309311 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76dj\" (UniqueName: \"kubernetes.io/projected/92d1c354-a475-41fc-832e-795cdff274d4-kube-api-access-v76dj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309311 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309772 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309706    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.309772 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.309722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.311785 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.311753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92d1c354-a475-41fc-832e-795cdff274d4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.312608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.312586    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92d1c354-a475-41fc-832e-795cdff274d4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.318568 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.318546    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76dj\" (UniqueName: \"kubernetes.io/projected/92d1c354-a475-41fc-832e-795cdff274d4-kube-api-access-v76dj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-x99mk\" (UID: \"92d1c354-a475-41fc-832e-795cdff274d4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.458568 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.458468    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:29.603519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.603485    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"]
Aug 03 16:14:29.606280 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:14:29.606251    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d1c354_a475_41fc_832e_795cdff274d4.slice/crio-81bd8077614746550f47bddbf8e737ea73a54d00fbf8b71d5c6456fbd60c035d WatchSource:0}: Error finding container 81bd8077614746550f47bddbf8e737ea73a54d00fbf8b71d5c6456fbd60c035d: Status 404 returned error can't find the container with id 81bd8077614746550f47bddbf8e737ea73a54d00fbf8b71d5c6456fbd60c035d
Aug 03 16:14:29.936806 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.936772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk" event={"ID":"92d1c354-a475-41fc-832e-795cdff274d4","Type":"ContainerStarted","Data":"170e4a1b4f42dbc51eaa5035d96dadf5d80594ce00c95efd3994ad96c3f436e8"}
Aug 03 16:14:29.936943 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.936827    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk" event={"ID":"92d1c354-a475-41fc-832e-795cdff274d4","Type":"ContainerStarted","Data":"81bd8077614746550f47bddbf8e737ea73a54d00fbf8b71d5c6456fbd60c035d"}
Aug 03 16:14:29.956670 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:29.956627    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk" podStartSLOduration=0.724405223 podStartE2EDuration="956.611771ms" totalImagesPullingTime="232.206548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:14:29.954115799 +0000 UTC m=+629.503402184" watchObservedRunningTime="2026-08-03 16:14:29.956611771 +0000 UTC m=+629.505898086"
Aug 03 16:14:39.459260 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:39.459221    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:39.459677 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:39.459550    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:39.471906 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:39.471884    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:39.994029 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:39.994001    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-x99mk"
Aug 03 16:14:51.867658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:51.867608    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:14:58.767474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:14:58.767437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:15:00.137109 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:00.137073    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"]
Aug 03 16:15:00.140662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:00.140644    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"
Aug 03 16:15:00.143537 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:00.143503    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:15:00.147907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:00.147870    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"]
Aug 03 16:15:00.152667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:00.152648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"
Aug 03 16:15:00.287438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:00.287414    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"]
Aug 03 16:15:00.289451 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:15:00.289422    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a021c2_2ae5_4784_94a8_709e17797fc8.slice/crio-638da712e777715a2bdfc4005a52093def9afc88c0fc176314ca59a846204972 WatchSource:0}: Error finding container 638da712e777715a2bdfc4005a52093def9afc88c0fc176314ca59a846204972: Status 404 returned error can't find the container with id 638da712e777715a2bdfc4005a52093def9afc88c0fc176314ca59a846204972
Aug 03 16:15:01.079350 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:01.079319    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l" event={"ID":"12a021c2-2ae5-4784-94a8-709e17797fc8","Type":"ContainerStarted","Data":"638da712e777715a2bdfc4005a52093def9afc88c0fc176314ca59a846204972"}
Aug 03 16:15:03.092982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:03.092944    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l" event={"ID":"12a021c2-2ae5-4784-94a8-709e17797fc8","Type":"ContainerStarted","Data":"07a79442c934f7cdaf599a03c2cb3eb22e72b1e9c5fe078d9a1a8e97e681ed61"}
Aug 03 16:15:03.109950 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:03.109901    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l" podStartSLOduration=0.71004871 podStartE2EDuration="3.109885229s" totalImagesPullingTime="2.399836519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:15:03.109095617 +0000 UTC m=+662.658381933" watchObservedRunningTime="2026-08-03 16:15:03.109885229 +0000 UTC m=+662.659171544"
Aug 03 16:15:04.099367 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:04.099333    2560 generic.go:356] "Generic (PLEG): container finished" podID="12a021c2-2ae5-4784-94a8-709e17797fc8" containerID="07a79442c934f7cdaf599a03c2cb3eb22e72b1e9c5fe078d9a1a8e97e681ed61" exitCode=0
Aug 03 16:15:04.099856 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:04.099381    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l" event={"ID":"12a021c2-2ae5-4784-94a8-709e17797fc8","Type":"ContainerDied","Data":"07a79442c934f7cdaf599a03c2cb3eb22e72b1e9c5fe078d9a1a8e97e681ed61"}
Aug 03 16:15:05.249232 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:05.249205    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"
Aug 03 16:15:06.109385 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:06.109355    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"
Aug 03 16:15:06.109385 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:06.109361    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l" event={"ID":"12a021c2-2ae5-4784-94a8-709e17797fc8","Type":"ContainerDied","Data":"638da712e777715a2bdfc4005a52093def9afc88c0fc176314ca59a846204972"}
Aug 03 16:15:06.109700 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:06.109405    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="638da712e777715a2bdfc4005a52093def9afc88c0fc176314ca59a846204972"
Aug 03 16:15:09.589576 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:09.589501    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:15:39.886517 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:39.886427    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:15:55.379070 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:55.378981    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:15:55.440109 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:55.440071    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:15:59.086079 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:59.086040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:15:59.175261 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:15:59.175221    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:16:00.464259 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:00.464216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:16:00.566007 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:00.565975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:16:46.837116 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:46.837081    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69978859c5-h5jqd"]
Aug 03 16:16:46.837674 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:46.837309    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69978859c5-h5jqd" podUID="a639e974-dd0b-4fe2-9fb5-f83b92e177e7" containerName="manager" containerID="cri-o://afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c" gracePeriod=30
Aug 03 16:16:47.096191 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.096129    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:16:47.137722 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.137689    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-webhook-cert\") pod \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") "
Aug 03 16:16:47.137882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.137773    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mppxc\" (UniqueName: \"kubernetes.io/projected/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-kube-api-access-mppxc\") pod \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\" (UID: \"a639e974-dd0b-4fe2-9fb5-f83b92e177e7\") "
Aug 03 16:16:47.139757 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.139731    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a639e974-dd0b-4fe2-9fb5-f83b92e177e7" (UID: "a639e974-dd0b-4fe2-9fb5-f83b92e177e7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:16:47.139934 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.139912    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-kube-api-access-mppxc" (OuterVolumeSpecName: "kube-api-access-mppxc") pod "a639e974-dd0b-4fe2-9fb5-f83b92e177e7" (UID: "a639e974-dd0b-4fe2-9fb5-f83b92e177e7"). InnerVolumeSpecName "kube-api-access-mppxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:16:47.238955 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.238918    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-webhook-cert\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:16:47.238955 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.238946    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mppxc\" (UniqueName: \"kubernetes.io/projected/a639e974-dd0b-4fe2-9fb5-f83b92e177e7-kube-api-access-mppxc\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:16:47.602379 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.602343    2560 generic.go:356] "Generic (PLEG): container finished" podID="a639e974-dd0b-4fe2-9fb5-f83b92e177e7" containerID="afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c" exitCode=0
Aug 03 16:16:47.602562 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.602390    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-h5jqd"
Aug 03 16:16:47.602562 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.602425    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-h5jqd" event={"ID":"a639e974-dd0b-4fe2-9fb5-f83b92e177e7","Type":"ContainerDied","Data":"afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c"}
Aug 03 16:16:47.602562 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.602475    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-h5jqd" event={"ID":"a639e974-dd0b-4fe2-9fb5-f83b92e177e7","Type":"ContainerDied","Data":"134f3e0915c4148c4de24431a9622ca8fe9daabb866ff15d106496fe03e866ec"}
Aug 03 16:16:47.602562 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.602497    2560 scope.go:117] "RemoveContainer" containerID="afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c"
Aug 03 16:16:47.611568 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.611550    2560 scope.go:117] "RemoveContainer" containerID="afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c"
Aug 03 16:16:47.611814 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:16:47.611795    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c\": container with ID starting with afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c not found: ID does not exist" containerID="afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c"
Aug 03 16:16:47.611873 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.611822    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c"} err="failed to get container status \"afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c\": rpc error: code = NotFound desc = could not find container \"afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c\": container with ID starting with afe1eef74a82fdaa811f6c0310f3c1d08ff912092e35c74eb013549e47487b6c not found: ID does not exist"
Aug 03 16:16:47.623802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.623782    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69978859c5-h5jqd"]
Aug 03 16:16:47.626403 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:47.626378    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69978859c5-h5jqd"]
Aug 03 16:16:48.987092 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:16:48.987058    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a639e974-dd0b-4fe2-9fb5-f83b92e177e7" path="/var/lib/kubelet/pods/a639e974-dd0b-4fe2-9fb5-f83b92e177e7/volumes"
Aug 03 16:17:46.482637 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:46.482600    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:17:46.573112 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:46.573083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:17:48.446771 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.446734    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69978859c5-dqcvg"]
Aug 03 16:17:48.447143 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.447087    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a639e974-dd0b-4fe2-9fb5-f83b92e177e7" containerName="manager"
Aug 03 16:17:48.447143 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.447098    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a639e974-dd0b-4fe2-9fb5-f83b92e177e7" containerName="manager"
Aug 03 16:17:48.447143 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.447117    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12a021c2-2ae5-4784-94a8-709e17797fc8" containerName="cleanup"
Aug 03 16:17:48.447143 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.447123    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="12a021c2-2ae5-4784-94a8-709e17797fc8" containerName="cleanup"
Aug 03 16:17:48.447278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.447186    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="12a021c2-2ae5-4784-94a8-709e17797fc8" containerName="cleanup"
Aug 03 16:17:48.447278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.447195    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a639e974-dd0b-4fe2-9fb5-f83b92e177e7" containerName="manager"
Aug 03 16:17:48.450320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.450303    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.453120 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.453096    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kmz6t\""
Aug 03 16:17:48.454539 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.454511    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 03 16:17:48.454693 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.454673    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 03 16:17:48.460585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.460560    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69978859c5-dqcvg"]
Aug 03 16:17:48.577140 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.577103    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-webhook-cert\") pod \"maas-controller-69978859c5-dqcvg\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") " pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.577308 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.577154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7z9k\" (UniqueName: \"kubernetes.io/projected/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-kube-api-access-k7z9k\") pod \"maas-controller-69978859c5-dqcvg\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") " pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.678621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.678583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-webhook-cert\") pod \"maas-controller-69978859c5-dqcvg\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") " pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.678788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.678638    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7z9k\" (UniqueName: \"kubernetes.io/projected/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-kube-api-access-k7z9k\") pod \"maas-controller-69978859c5-dqcvg\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") " pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.681182 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.681159    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-webhook-cert\") pod \"maas-controller-69978859c5-dqcvg\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") " pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.688836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.688814    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7z9k\" (UniqueName: \"kubernetes.io/projected/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-kube-api-access-k7z9k\") pod \"maas-controller-69978859c5-dqcvg\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") " pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.762435 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.762353    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:48.896977 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.896950    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69978859c5-dqcvg"]
Aug 03 16:17:48.898704 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:17:48.898679    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849bd956_96e1_4197_bd2b_b4e0b9ee45b9.slice/crio-e5dde871bdcb2121f362578ff53ebc01bb53b33fbddc605c31ec12cbaf59506e WatchSource:0}: Error finding container e5dde871bdcb2121f362578ff53ebc01bb53b33fbddc605c31ec12cbaf59506e: Status 404 returned error can't find the container with id e5dde871bdcb2121f362578ff53ebc01bb53b33fbddc605c31ec12cbaf59506e
Aug 03 16:17:48.899982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:48.899949    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:17:49.891581 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:49.891519    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-dqcvg" event={"ID":"849bd956-96e1-4197-bd2b-b4e0b9ee45b9","Type":"ContainerStarted","Data":"a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d"}
Aug 03 16:17:49.892010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:49.891604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-dqcvg" event={"ID":"849bd956-96e1-4197-bd2b-b4e0b9ee45b9","Type":"ContainerStarted","Data":"e5dde871bdcb2121f362578ff53ebc01bb53b33fbddc605c31ec12cbaf59506e"}
Aug 03 16:17:49.892010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:49.891694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:17:49.910400 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:17:49.910324    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69978859c5-dqcvg" podStartSLOduration=1.474230296 podStartE2EDuration="1.910308506s" totalImagesPullingTime="436.07821ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:17:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:17:49.909805632 +0000 UTC m=+829.459091949" watchObservedRunningTime="2026-08-03 16:17:49.910308506 +0000 UTC m=+829.459594823"
Aug 03 16:18:00.899946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:00.899913    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:18:05.450212 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.450171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:18:05.562675 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.562643    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69978859c5-dqcvg"]
Aug 03 16:18:05.562858 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.562823    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69978859c5-dqcvg" podUID="849bd956-96e1-4197-bd2b-b4e0b9ee45b9" containerName="manager" containerID="cri-o://a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d" gracePeriod=30
Aug 03 16:18:05.812855 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.812828    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:18:05.929934 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.929904    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-webhook-cert\") pod \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") "
Aug 03 16:18:05.930126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.929977    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7z9k\" (UniqueName: \"kubernetes.io/projected/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-kube-api-access-k7z9k\") pod \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\" (UID: \"849bd956-96e1-4197-bd2b-b4e0b9ee45b9\") "
Aug 03 16:18:05.932157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.932117    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-kube-api-access-k7z9k" (OuterVolumeSpecName: "kube-api-access-k7z9k") pod "849bd956-96e1-4197-bd2b-b4e0b9ee45b9" (UID: "849bd956-96e1-4197-bd2b-b4e0b9ee45b9"). InnerVolumeSpecName "kube-api-access-k7z9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:18:05.932157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.932124    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "849bd956-96e1-4197-bd2b-b4e0b9ee45b9" (UID: "849bd956-96e1-4197-bd2b-b4e0b9ee45b9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:18:05.971360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.971326    2560 generic.go:356] "Generic (PLEG): container finished" podID="849bd956-96e1-4197-bd2b-b4e0b9ee45b9" containerID="a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d" exitCode=0
Aug 03 16:18:05.971521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.971372    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-dqcvg"
Aug 03 16:18:05.971521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.971407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-dqcvg" event={"ID":"849bd956-96e1-4197-bd2b-b4e0b9ee45b9","Type":"ContainerDied","Data":"a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d"}
Aug 03 16:18:05.971521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.971461    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-dqcvg" event={"ID":"849bd956-96e1-4197-bd2b-b4e0b9ee45b9","Type":"ContainerDied","Data":"e5dde871bdcb2121f362578ff53ebc01bb53b33fbddc605c31ec12cbaf59506e"}
Aug 03 16:18:05.971521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.971490    2560 scope.go:117] "RemoveContainer" containerID="a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d"
Aug 03 16:18:05.981167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.981146    2560 scope.go:117] "RemoveContainer" containerID="a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d"
Aug 03 16:18:05.982925 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:18:05.982864    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d\": container with ID starting with a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d not found: ID does not exist" containerID="a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d"
Aug 03 16:18:05.982925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.982907    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d"} err="failed to get container status \"a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d\": rpc error: code = NotFound desc = could not find container \"a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d\": container with ID starting with a60cafb0cab330d31d702e6db3b9f210e8e92be5d05fb1854f313432ea80768d not found: ID does not exist"
Aug 03 16:18:05.994938 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:05.994913    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69978859c5-dqcvg"]
Aug 03 16:18:06.000647 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.000625    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69978859c5-dqcvg"]
Aug 03 16:18:06.031278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.031256    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-webhook-cert\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:18:06.031278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.031276    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7z9k\" (UniqueName: \"kubernetes.io/projected/849bd956-96e1-4197-bd2b-b4e0b9ee45b9-kube-api-access-k7z9k\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:18:06.723212 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.723177    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:18:06.852476 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.852437    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69978859c5-k72h6"]
Aug 03 16:18:06.852887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.852869    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849bd956-96e1-4197-bd2b-b4e0b9ee45b9" containerName="manager"
Aug 03 16:18:06.853003 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.852889    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="849bd956-96e1-4197-bd2b-b4e0b9ee45b9" containerName="manager"
Aug 03 16:18:06.853055 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.853012    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="849bd956-96e1-4197-bd2b-b4e0b9ee45b9" containerName="manager"
Aug 03 16:18:06.857217 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.857194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:06.859983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.859966    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 03 16:18:06.861608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.861579    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 03 16:18:06.861892 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.861876    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kmz6t\""
Aug 03 16:18:06.864804 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.864285    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69978859c5-k72h6"]
Aug 03 16:18:06.938370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.938338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96428d5f-8e38-45e4-b12b-eb7d7e6a6853-webhook-cert\") pod \"maas-controller-69978859c5-k72h6\" (UID: \"96428d5f-8e38-45e4-b12b-eb7d7e6a6853\") " pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:06.938370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.938375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5w4\" (UniqueName: \"kubernetes.io/projected/96428d5f-8e38-45e4-b12b-eb7d7e6a6853-kube-api-access-zg5w4\") pod \"maas-controller-69978859c5-k72h6\" (UID: \"96428d5f-8e38-45e4-b12b-eb7d7e6a6853\") " pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:06.986004 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:06.985929    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849bd956-96e1-4197-bd2b-b4e0b9ee45b9" path="/var/lib/kubelet/pods/849bd956-96e1-4197-bd2b-b4e0b9ee45b9/volumes"
Aug 03 16:18:07.038902 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.038866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96428d5f-8e38-45e4-b12b-eb7d7e6a6853-webhook-cert\") pod \"maas-controller-69978859c5-k72h6\" (UID: \"96428d5f-8e38-45e4-b12b-eb7d7e6a6853\") " pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:07.039098 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.038906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5w4\" (UniqueName: \"kubernetes.io/projected/96428d5f-8e38-45e4-b12b-eb7d7e6a6853-kube-api-access-zg5w4\") pod \"maas-controller-69978859c5-k72h6\" (UID: \"96428d5f-8e38-45e4-b12b-eb7d7e6a6853\") " pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:07.041368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.041343    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96428d5f-8e38-45e4-b12b-eb7d7e6a6853-webhook-cert\") pod \"maas-controller-69978859c5-k72h6\" (UID: \"96428d5f-8e38-45e4-b12b-eb7d7e6a6853\") " pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:07.048793 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.048772    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5w4\" (UniqueName: \"kubernetes.io/projected/96428d5f-8e38-45e4-b12b-eb7d7e6a6853-kube-api-access-zg5w4\") pod \"maas-controller-69978859c5-k72h6\" (UID: \"96428d5f-8e38-45e4-b12b-eb7d7e6a6853\") " pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:07.169129 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.169091    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:07.305126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.305091    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69978859c5-k72h6"]
Aug 03 16:18:07.308467 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:18:07.308443    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96428d5f_8e38_45e4_b12b_eb7d7e6a6853.slice/crio-0d202de60afe5a325f8a9adb6a2e609efa0ee6399a0c51da6e3aa983a54f78ce WatchSource:0}: Error finding container 0d202de60afe5a325f8a9adb6a2e609efa0ee6399a0c51da6e3aa983a54f78ce: Status 404 returned error can't find the container with id 0d202de60afe5a325f8a9adb6a2e609efa0ee6399a0c51da6e3aa983a54f78ce
Aug 03 16:18:07.983584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.983472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-k72h6" event={"ID":"96428d5f-8e38-45e4-b12b-eb7d7e6a6853","Type":"ContainerStarted","Data":"28c98b936cafbc283f466d72c2deb10e959e936c4859d1620f271dc83f99c3b4"}
Aug 03 16:18:07.983584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.983550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69978859c5-k72h6" event={"ID":"96428d5f-8e38-45e4-b12b-eb7d7e6a6853","Type":"ContainerStarted","Data":"0d202de60afe5a325f8a9adb6a2e609efa0ee6399a0c51da6e3aa983a54f78ce"}
Aug 03 16:18:07.983584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:07.983581    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:08.003942 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:08.003891    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69978859c5-k72h6" podStartSLOduration=1.619153956 podStartE2EDuration="2.003876612s" totalImagesPullingTime="384.722656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:18:08.002012024 +0000 UTC m=+847.551298339" watchObservedRunningTime="2026-08-03 16:18:08.003876612 +0000 UTC m=+847.553162966"
Aug 03 16:18:12.268683 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:12.268648    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:18:18.990403 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:18.990374    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69978859c5-k72h6"
Aug 03 16:18:57.653300 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:18:57.653225    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:00.938539 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:00.938495    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:19:00.940733 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:00.940708    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:19:09.659285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:09.659249    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:09.760031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:09.760000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:09.913014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:09.912939    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:15.959483 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:15.959446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:16.064593 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:16.064555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:16.162264 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:16.162229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:16.371138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:16.371100    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:22.358326 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:22.358293    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:19:22.435141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:19:22.435108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:20:35.127572 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:20:35.127523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:24:00.970554 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:24:00.970510    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:24:00.974090 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:24:00.974061    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:24:20.228676 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:24:20.228632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:25:12.920550 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:25:12.920442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:26:18.043367 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:26:18.043320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:27:06.329248 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:27:06.329199    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:27:06.385954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:27:06.385921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:28:32.490935 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:28:32.490898    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:28:46.293919 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:28:46.293865    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:29:00.999013 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:29:00.998981    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:29:01.004711 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:29:01.004685    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:29:11.387147 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:29:11.387114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:29:12.691541 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:29:12.691495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:30:00.134386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.134352    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"]
Aug 03 16:30:00.137512 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.137496    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"
Aug 03 16:30:00.140198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.140180    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:30:00.146265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.146237    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"]
Aug 03 16:30:00.149103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.149085    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"
Aug 03 16:30:00.288400 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.288372    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"]
Aug 03 16:30:00.291175 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:30:00.291145    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f47945_d6d7_47b4_a28b_2c848e239965.slice/crio-fa89362fdba4646e859d330be237b24f43b906b5cc42921ff0a291fe8379497e WatchSource:0}: Error finding container fa89362fdba4646e859d330be237b24f43b906b5cc42921ff0a291fe8379497e: Status 404 returned error can't find the container with id fa89362fdba4646e859d330be237b24f43b906b5cc42921ff0a291fe8379497e
Aug 03 16:30:00.381402 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:00.381365    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz" event={"ID":"65f47945-d6d7-47b4-a28b-2c848e239965","Type":"ContainerStarted","Data":"fa89362fdba4646e859d330be237b24f43b906b5cc42921ff0a291fe8379497e"}
Aug 03 16:30:01.391167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:01.391130    2560 generic.go:356] "Generic (PLEG): container finished" podID="65f47945-d6d7-47b4-a28b-2c848e239965" containerID="e1555cf9cb4d2b2dea752ecbf12e37559419048db92a702b27fb56e33505877f" exitCode=0
Aug 03 16:30:01.391620 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:01.391224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz" event={"ID":"65f47945-d6d7-47b4-a28b-2c848e239965","Type":"ContainerDied","Data":"e1555cf9cb4d2b2dea752ecbf12e37559419048db92a702b27fb56e33505877f"}
Aug 03 16:30:02.538743 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:02.538715    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"
Aug 03 16:30:03.403063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:03.403029    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz"
Aug 03 16:30:03.403245 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:03.403069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29762910-nqnkz" event={"ID":"65f47945-d6d7-47b4-a28b-2c848e239965","Type":"ContainerDied","Data":"fa89362fdba4646e859d330be237b24f43b906b5cc42921ff0a291fe8379497e"}
Aug 03 16:30:03.403245 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:30:03.403154    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa89362fdba4646e859d330be237b24f43b906b5cc42921ff0a291fe8379497e"
Aug 03 16:32:33.123947 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:32:33.123918    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:32:36.619826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:32:36.619788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:02.535633 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:02.535599    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:05.601751 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:05.601718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:05.702854 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:05.702819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:06.121857 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:06.121828    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:12.366721 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:12.366685    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:12.435200 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:12.435167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:12.610694 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:12.610662    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:12.939778 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:12.939742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:13.104225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:13.104188    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:15.602110 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:15.602081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:15.695112 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:15.695081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:16.692690 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.692653    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"]
Aug 03 16:33:16.693188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.693167    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f47945-d6d7-47b4-a28b-2c848e239965" containerName="cleanup"
Aug 03 16:33:16.693253 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.693192    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f47945-d6d7-47b4-a28b-2c848e239965" containerName="cleanup"
Aug 03 16:33:16.693307 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.693260    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f47945-d6d7-47b4-a28b-2c848e239965" containerName="cleanup"
Aug 03 16:33:16.696471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.696450    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.699024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.699007    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-77570ccf-serving-cert\""
Aug 03 16:33:16.704926 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.704893    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"]
Aug 03 16:33:16.758070 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.758036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:16.776957 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.776921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zrsg\" (UniqueName: \"kubernetes.io/projected/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-kube-api-access-7zrsg\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.777126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.776974    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.777126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.777015    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-openshift-service-ca\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.878269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.878227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zrsg\" (UniqueName: \"kubernetes.io/projected/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-kube-api-access-7zrsg\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.878269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.878272    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.878575 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.878290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-openshift-service-ca\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.878575 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:33:16.878402    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-serving-cert: secret "maas-api-e2e-ait-77570ccf-serving-cert" not found
Aug 03 16:33:16.878575 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:33:16.878489    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls podName:9ada4dc7-c936-47a2-9fc5-0fa671d845a2 nodeName:}" failed. No retries permitted until 2026-08-03 16:33:17.378469081 +0000 UTC m=+1756.927755376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls") pod "maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" (UID: "9ada4dc7-c936-47a2-9fc5-0fa671d845a2") : secret "maas-api-e2e-ait-77570ccf-serving-cert" not found
Aug 03 16:33:16.879015 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.878996    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-openshift-service-ca\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:16.887790 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:16.887761    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zrsg\" (UniqueName: \"kubernetes.io/projected/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-kube-api-access-7zrsg\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:17.072077 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.072050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:17.266068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.266032    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:17.359128 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.359039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:17.363744 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.363713    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"]
Aug 03 16:33:17.367245 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.367227    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.369999 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.369975    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-77570ccf\""
Aug 03 16:33:17.377232 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.377207    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"]
Aug 03 16:33:17.383424 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.383396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:17.386141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.386115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls\") pod \"maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:17.386549 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.386516    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"]
Aug 03 16:33:17.395843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.395803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.399731 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.399706    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-77570ccf-dockercfg-gq2m7\""
Aug 03 16:33:17.404461 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.404430    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"]
Aug 03 16:33:17.484210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.484169    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.484210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.484216    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9da7823-0d57-448a-bcb6-faeb39a1fda4-plugins-config-volume\") pod \"payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") " pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.484449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.484250    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64shd\" (UniqueName: \"kubernetes.io/projected/b9da7823-0d57-448a-bcb6-faeb39a1fda4-kube-api-access-64shd\") pod \"payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") " pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.484449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.484385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fh59\" (UniqueName: \"kubernetes.io/projected/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-kube-api-access-9fh59\") pod \"payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.585269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.585222    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fh59\" (UniqueName: \"kubernetes.io/projected/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-kube-api-access-9fh59\") pod \"payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.585482 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.585304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.585482 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.585331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9da7823-0d57-448a-bcb6-faeb39a1fda4-plugins-config-volume\") pod \"payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") " pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.585482 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.585363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64shd\" (UniqueName: \"kubernetes.io/projected/b9da7823-0d57-448a-bcb6-faeb39a1fda4-kube-api-access-64shd\") pod \"payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") " pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.586107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.586084    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9da7823-0d57-448a-bcb6-faeb39a1fda4-plugins-config-volume\") pod \"payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") " pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.586154 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.586109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.595285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.595255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64shd\" (UniqueName: \"kubernetes.io/projected/b9da7823-0d57-448a-bcb6-faeb39a1fda4-kube-api-access-64shd\") pod \"payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") " pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.595738 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.595713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fh59\" (UniqueName: \"kubernetes.io/projected/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-kube-api-access-9fh59\") pod \"payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.608407 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.608379    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:17.680271 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.680239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:17.713345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.713194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:17.764809 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.763922    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"]
Aug 03 16:33:17.768667 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:17.767051    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ada4dc7_c936_47a2_9fc5_0fa671d845a2.slice/crio-c2024ddd235459942955da38f6e1098ca71f10cf3fdf5f9cdf8c009d67d66ffc WatchSource:0}: Error finding container c2024ddd235459942955da38f6e1098ca71f10cf3fdf5f9cdf8c009d67d66ffc: Status 404 returned error can't find the container with id c2024ddd235459942955da38f6e1098ca71f10cf3fdf5f9cdf8c009d67d66ffc
Aug 03 16:33:17.770017 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.769440    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:33:17.854224 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.854181    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"]
Aug 03 16:33:17.858171 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:17.858140    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b699cb8_f0df_4a6e_8970_2c5b3a1971b6.slice/crio-9c7567dde765744a7662a021949ac2b5a8d7e05fb29591e623b79225247bf416 WatchSource:0}: Error finding container 9c7567dde765744a7662a021949ac2b5a8d7e05fb29591e623b79225247bf416: Status 404 returned error can't find the container with id 9c7567dde765744a7662a021949ac2b5a8d7e05fb29591e623b79225247bf416
Aug 03 16:33:17.885874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:17.885847    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"]
Aug 03 16:33:17.890915 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:17.890885    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9da7823_0d57_448a_bcb6_faeb39a1fda4.slice/crio-1374da0482f5b42e0dcf5b4cba4046bbd96e104b309cf1eea87ac66737ef3236 WatchSource:0}: Error finding container 1374da0482f5b42e0dcf5b4cba4046bbd96e104b309cf1eea87ac66737ef3236: Status 404 returned error can't find the container with id 1374da0482f5b42e0dcf5b4cba4046bbd96e104b309cf1eea87ac66737ef3236
Aug 03 16:33:18.346499 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.346460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz" event={"ID":"b9da7823-0d57-448a-bcb6-faeb39a1fda4","Type":"ContainerStarted","Data":"df007746498d84dabd051e9907e66e9a99755795494aaf7de82f3fa47762df52"}
Aug 03 16:33:18.346720 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.346522    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz" event={"ID":"b9da7823-0d57-448a-bcb6-faeb39a1fda4","Type":"ContainerStarted","Data":"1374da0482f5b42e0dcf5b4cba4046bbd96e104b309cf1eea87ac66737ef3236"}
Aug 03 16:33:18.346720 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.346579    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:18.348179 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.348147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl" event={"ID":"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6","Type":"ContainerStarted","Data":"005eaaf74e27fe02797260ba66127b5c92f70d1f0dccf70ab7178d26dfa51186"}
Aug 03 16:33:18.348294 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.348205    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl" event={"ID":"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6","Type":"ContainerStarted","Data":"9c7567dde765744a7662a021949ac2b5a8d7e05fb29591e623b79225247bf416"}
Aug 03 16:33:18.348294 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.348279    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:18.349487 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.349462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" event={"ID":"9ada4dc7-c936-47a2-9fc5-0fa671d845a2","Type":"ContainerStarted","Data":"c2024ddd235459942955da38f6e1098ca71f10cf3fdf5f9cdf8c009d67d66ffc"}
Aug 03 16:33:18.363491 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.363440    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz" podStartSLOduration=1.363426184 podStartE2EDuration="1.363426184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:18.362621334 +0000 UTC m=+1757.911907654" watchObservedRunningTime="2026-08-03 16:33:18.363426184 +0000 UTC m=+1757.912712480"
Aug 03 16:33:18.379857 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:18.379794    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl" podStartSLOduration=1.379771514 podStartE2EDuration="1.379771514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:18.377537325 +0000 UTC m=+1757.926823684" watchObservedRunningTime="2026-08-03 16:33:18.379771514 +0000 UTC m=+1757.929057830"
Aug 03 16:33:19.356159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:19.356121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" event={"ID":"9ada4dc7-c936-47a2-9fc5-0fa671d845a2","Type":"ContainerStarted","Data":"0d575138520edfb1bab66d7e9dc351ba2890a3b1490f1b00897405f0619f651d"}
Aug 03 16:33:19.376607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:19.376547    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" podStartSLOduration=2.833815658 podStartE2EDuration="3.376516521s" totalImagesPullingTime="542.700863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:19.374558227 +0000 UTC m=+1758.923844545" watchObservedRunningTime="2026-08-03 16:33:19.376516521 +0000 UTC m=+1758.925802837"
Aug 03 16:33:20.361621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:20.361589    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:26.379051 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:26.379022    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:29.358576 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:29.358546    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:29.359068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:29.358647    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:41.176432 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.176395    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"]
Aug 03 16:33:41.179978 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.179960    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:41.182447 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.182426    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:33:41.187073 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.187047    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"]
Aug 03 16:33:41.310422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.310383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2095c01e-8c97-4152-8862-9846bf89e022-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz\" (UID: \"2095c01e-8c97-4152-8862-9846bf89e022\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:41.410967 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.410931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2095c01e-8c97-4152-8862-9846bf89e022-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz\" (UID: \"2095c01e-8c97-4152-8862-9846bf89e022\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:41.411585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.411567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2095c01e-8c97-4152-8862-9846bf89e022-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz\" (UID: \"2095c01e-8c97-4152-8862-9846bf89e022\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:41.492726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.492638    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:41.635661 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:41.635628    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"]
Aug 03 16:33:41.636855 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:41.636828    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2095c01e_8c97_4152_8862_9846bf89e022.slice/crio-8a52a9a9b4714540568729a854b55c86d5b424ff606871cfb3ba4993ec970945 WatchSource:0}: Error finding container 8a52a9a9b4714540568729a854b55c86d5b424ff606871cfb3ba4993ec970945: Status 404 returned error can't find the container with id 8a52a9a9b4714540568729a854b55c86d5b424ff606871cfb3ba4993ec970945
Aug 03 16:33:42.486028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:42.485989    2560 generic.go:356] "Generic (PLEG): container finished" podID="2095c01e-8c97-4152-8862-9846bf89e022" containerID="dfa5786db48a2a7e180e4e8840758f265d8c9fdea9d73fbaf0ba160864ddee48" exitCode=0
Aug 03 16:33:42.486422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:42.486079    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz" event={"ID":"2095c01e-8c97-4152-8862-9846bf89e022","Type":"ContainerDied","Data":"dfa5786db48a2a7e180e4e8840758f265d8c9fdea9d73fbaf0ba160864ddee48"}
Aug 03 16:33:42.486422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:42.486140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz" event={"ID":"2095c01e-8c97-4152-8862-9846bf89e022","Type":"ContainerStarted","Data":"8a52a9a9b4714540568729a854b55c86d5b424ff606871cfb3ba4993ec970945"}
Aug 03 16:33:43.635189 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:43.635163    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:43.733049 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:43.732997    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2095c01e-8c97-4152-8862-9846bf89e022-maas-api-service-ca\") pod \"2095c01e-8c97-4152-8862-9846bf89e022\" (UID: \"2095c01e-8c97-4152-8862-9846bf89e022\") "
Aug 03 16:33:43.733411 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:43.733384    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2095c01e-8c97-4152-8862-9846bf89e022-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2095c01e-8c97-4152-8862-9846bf89e022" (UID: "2095c01e-8c97-4152-8862-9846bf89e022"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:33:43.834303 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:43.834261    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2095c01e-8c97-4152-8862-9846bf89e022-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:44.497261 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:44.497224    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"
Aug 03 16:33:44.497445 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:44.497259    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz" event={"ID":"2095c01e-8c97-4152-8862-9846bf89e022","Type":"ContainerDied","Data":"8a52a9a9b4714540568729a854b55c86d5b424ff606871cfb3ba4993ec970945"}
Aug 03 16:33:44.497445 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:44.497311    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a52a9a9b4714540568729a854b55c86d5b424ff606871cfb3ba4993ec970945"
Aug 03 16:33:47.211572 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.211515    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"]
Aug 03 16:33:47.212163 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.211798    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" podUID="9ada4dc7-c936-47a2-9fc5-0fa671d845a2" containerName="maas-api" containerID="cri-o://0d575138520edfb1bab66d7e9dc351ba2890a3b1490f1b00897405f0619f651d" gracePeriod=30
Aug 03 16:33:47.249943 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.249911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:47.278948 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.278911    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"]
Aug 03 16:33:47.279156 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.279128    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz" podUID="b9da7823-0d57-448a-bcb6-faeb39a1fda4" containerName="payload-processing" containerID="cri-o://df007746498d84dabd051e9907e66e9a99755795494aaf7de82f3fa47762df52" gracePeriod=30
Aug 03 16:33:47.288325 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.288296    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"]
Aug 03 16:33:47.288563 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.288521    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl" podUID="2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" containerName="payload-pre-processing" containerID="cri-o://005eaaf74e27fe02797260ba66127b5c92f70d1f0dccf70ab7178d26dfa51186" gracePeriod=30
Aug 03 16:33:47.350121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.350080    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:47.429719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.429548    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"]
Aug 03 16:33:47.431823 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.431760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:47.432112 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.432078    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-77570ccf-f2bhz"]
Aug 03 16:33:47.498836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.498725    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:47.515760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.515628    2560 generic.go:356] "Generic (PLEG): container finished" podID="9ada4dc7-c936-47a2-9fc5-0fa671d845a2" containerID="0d575138520edfb1bab66d7e9dc351ba2890a3b1490f1b00897405f0619f651d" exitCode=0
Aug 03 16:33:47.515760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.515696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" event={"ID":"9ada4dc7-c936-47a2-9fc5-0fa671d845a2","Type":"ContainerDied","Data":"0d575138520edfb1bab66d7e9dc351ba2890a3b1490f1b00897405f0619f651d"}
Aug 03 16:33:47.517754 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.517731    2560 generic.go:356] "Generic (PLEG): container finished" podID="b9da7823-0d57-448a-bcb6-faeb39a1fda4" containerID="df007746498d84dabd051e9907e66e9a99755795494aaf7de82f3fa47762df52" exitCode=0
Aug 03 16:33:47.517894 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.517806    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz" event={"ID":"b9da7823-0d57-448a-bcb6-faeb39a1fda4","Type":"ContainerDied","Data":"df007746498d84dabd051e9907e66e9a99755795494aaf7de82f3fa47762df52"}
Aug 03 16:33:47.519639 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.519498    2560 generic.go:356] "Generic (PLEG): container finished" podID="2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" containerID="005eaaf74e27fe02797260ba66127b5c92f70d1f0dccf70ab7178d26dfa51186" exitCode=0
Aug 03 16:33:47.519639 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.519591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl" event={"ID":"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6","Type":"ContainerDied","Data":"005eaaf74e27fe02797260ba66127b5c92f70d1f0dccf70ab7178d26dfa51186"}
Aug 03 16:33:47.568780 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.568750    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:47.573003 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.572976    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:47.593480 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.593443    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:47.616095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.616064    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:47.669787 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.669755    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64shd\" (UniqueName: \"kubernetes.io/projected/b9da7823-0d57-448a-bcb6-faeb39a1fda4-kube-api-access-64shd\") pod \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") "
Aug 03 16:33:47.669983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.669796    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-plugins-config-volume\") pod \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") "
Aug 03 16:33:47.669983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.669838    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9da7823-0d57-448a-bcb6-faeb39a1fda4-plugins-config-volume\") pod \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\" (UID: \"b9da7823-0d57-448a-bcb6-faeb39a1fda4\") "
Aug 03 16:33:47.669983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.669887    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fh59\" (UniqueName: \"kubernetes.io/projected/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-kube-api-access-9fh59\") pod \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\" (UID: \"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6\") "
Aug 03 16:33:47.669983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.669916    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zrsg\" (UniqueName: \"kubernetes.io/projected/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-kube-api-access-7zrsg\") pod \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") "
Aug 03 16:33:47.670206 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.669986    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls\") pod \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") "
Aug 03 16:33:47.670206 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.670027    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-openshift-service-ca\") pod \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\" (UID: \"9ada4dc7-c936-47a2-9fc5-0fa671d845a2\") "
Aug 03 16:33:47.670304 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.670253    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" (UID: "2b699cb8-f0df-4a6e-8970-2c5b3a1971b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:33:47.670609 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.670568    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9da7823-0d57-448a-bcb6-faeb39a1fda4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9da7823-0d57-448a-bcb6-faeb39a1fda4" (UID: "b9da7823-0d57-448a-bcb6-faeb39a1fda4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:33:47.670717 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.670617    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ada4dc7-c936-47a2-9fc5-0fa671d845a2" (UID: "9ada4dc7-c936-47a2-9fc5-0fa671d845a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:33:47.672219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.672198    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-kube-api-access-9fh59" (OuterVolumeSpecName: "kube-api-access-9fh59") pod "2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" (UID: "2b699cb8-f0df-4a6e-8970-2c5b3a1971b6"). InnerVolumeSpecName "kube-api-access-9fh59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:33:47.672467 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.672406    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9da7823-0d57-448a-bcb6-faeb39a1fda4-kube-api-access-64shd" (OuterVolumeSpecName: "kube-api-access-64shd") pod "b9da7823-0d57-448a-bcb6-faeb39a1fda4" (UID: "b9da7823-0d57-448a-bcb6-faeb39a1fda4"). InnerVolumeSpecName "kube-api-access-64shd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:33:47.672467 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.672446    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ada4dc7-c936-47a2-9fc5-0fa671d845a2" (UID: "9ada4dc7-c936-47a2-9fc5-0fa671d845a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:33:47.672782 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.672763    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-kube-api-access-7zrsg" (OuterVolumeSpecName: "kube-api-access-7zrsg") pod "9ada4dc7-c936-47a2-9fc5-0fa671d845a2" (UID: "9ada4dc7-c936-47a2-9fc5-0fa671d845a2"). InnerVolumeSpecName "kube-api-access-7zrsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771487    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771521    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64shd\" (UniqueName: \"kubernetes.io/projected/b9da7823-0d57-448a-bcb6-faeb39a1fda4-kube-api-access-64shd\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771549    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771562    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9da7823-0d57-448a-bcb6-faeb39a1fda4-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771574    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fh59\" (UniqueName: \"kubernetes.io/projected/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6-kube-api-access-9fh59\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771586    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zrsg\" (UniqueName: \"kubernetes.io/projected/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-kube-api-access-7zrsg\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.771610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.771598    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ada4dc7-c936-47a2-9fc5-0fa671d845a2-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:33:47.815346 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:47.815316    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:48.061162 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.061130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:48.160186 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.160151    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:48.274041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.274010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:48.531901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.531805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz" event={"ID":"b9da7823-0d57-448a-bcb6-faeb39a1fda4","Type":"ContainerDied","Data":"1374da0482f5b42e0dcf5b4cba4046bbd96e104b309cf1eea87ac66737ef3236"}
Aug 03 16:33:48.531901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.531816    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"
Aug 03 16:33:48.532103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.531898    2560 scope.go:117] "RemoveContainer" containerID="df007746498d84dabd051e9907e66e9a99755795494aaf7de82f3fa47762df52"
Aug 03 16:33:48.533418 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.533218    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"
Aug 03 16:33:48.533418 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.533222    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl" event={"ID":"2b699cb8-f0df-4a6e-8970-2c5b3a1971b6","Type":"ContainerDied","Data":"9c7567dde765744a7662a021949ac2b5a8d7e05fb29591e623b79225247bf416"}
Aug 03 16:33:48.534719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.534693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47" event={"ID":"9ada4dc7-c936-47a2-9fc5-0fa671d845a2","Type":"ContainerDied","Data":"c2024ddd235459942955da38f6e1098ca71f10cf3fdf5f9cdf8c009d67d66ffc"}
Aug 03 16:33:48.534867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.534729    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"
Aug 03 16:33:48.544332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.544308    2560 scope.go:117] "RemoveContainer" containerID="005eaaf74e27fe02797260ba66127b5c92f70d1f0dccf70ab7178d26dfa51186"
Aug 03 16:33:48.558211 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.558179    2560 scope.go:117] "RemoveContainer" containerID="0d575138520edfb1bab66d7e9dc351ba2890a3b1490f1b00897405f0619f651d"
Aug 03 16:33:48.572320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.572291    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"]
Aug 03 16:33:48.578506 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.578479    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77570ccf-58f6b8b57c-zrh47"]
Aug 03 16:33:48.599247 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.599191    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"]
Aug 03 16:33:48.601177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.601149    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77570ccf-68bf5f5bc6-lz4hl"]
Aug 03 16:33:48.619573 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.619516    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"]
Aug 03 16:33:48.623274 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.623244    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-77570ccf-5dc7667c84-rjmjz"]
Aug 03 16:33:48.719910 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.719873    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:48.935113 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.935078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:48.987435 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.987403    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2095c01e-8c97-4152-8862-9846bf89e022" path="/var/lib/kubelet/pods/2095c01e-8c97-4152-8862-9846bf89e022/volumes"
Aug 03 16:33:48.987783 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.987770    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" path="/var/lib/kubelet/pods/2b699cb8-f0df-4a6e-8970-2c5b3a1971b6/volumes"
Aug 03 16:33:48.988117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.988104    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ada4dc7-c936-47a2-9fc5-0fa671d845a2" path="/var/lib/kubelet/pods/9ada4dc7-c936-47a2-9fc5-0fa671d845a2/volumes"
Aug 03 16:33:48.988416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:48.988405    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9da7823-0d57-448a-bcb6-faeb39a1fda4" path="/var/lib/kubelet/pods/b9da7823-0d57-448a-bcb6-faeb39a1fda4/volumes"
Aug 03 16:33:50.932269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:50.932235    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:51.009016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.008977    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"]
Aug 03 16:33:51.009594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009574    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9da7823-0d57-448a-bcb6-faeb39a1fda4" containerName="payload-processing"
Aug 03 16:33:51.009674 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009598    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9da7823-0d57-448a-bcb6-faeb39a1fda4" containerName="payload-processing"
Aug 03 16:33:51.009674 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009626    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2095c01e-8c97-4152-8862-9846bf89e022" containerName="revoke-keys"
Aug 03 16:33:51.009674 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009636    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2095c01e-8c97-4152-8862-9846bf89e022" containerName="revoke-keys"
Aug 03 16:33:51.009674 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009649    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" containerName="payload-pre-processing"
Aug 03 16:33:51.009674 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009661    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" containerName="payload-pre-processing"
Aug 03 16:33:51.009932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009680    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ada4dc7-c936-47a2-9fc5-0fa671d845a2" containerName="maas-api"
Aug 03 16:33:51.009932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009690    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ada4dc7-c936-47a2-9fc5-0fa671d845a2" containerName="maas-api"
Aug 03 16:33:51.009932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009790    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b699cb8-f0df-4a6e-8970-2c5b3a1971b6" containerName="payload-pre-processing"
Aug 03 16:33:51.009932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009803    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9da7823-0d57-448a-bcb6-faeb39a1fda4" containerName="payload-processing"
Aug 03 16:33:51.009932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009815    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2095c01e-8c97-4152-8862-9846bf89e022" containerName="revoke-keys"
Aug 03 16:33:51.009932 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.009825    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ada4dc7-c936-47a2-9fc5-0fa671d845a2" containerName="maas-api"
Aug 03 16:33:51.015024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.014998    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.017855 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.017836    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-51e95abc-serving-cert\""
Aug 03 16:33:51.022884 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.022462    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"]
Aug 03 16:33:51.052882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.052848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:51.103359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.103308    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfsdm\" (UniqueName: \"kubernetes.io/projected/f56428f5-ba84-4d49-bffd-16c932df24c6-kube-api-access-pfsdm\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.103583 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.103476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f56428f5-ba84-4d49-bffd-16c932df24c6-maas-api-tls\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.103658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.103618    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f56428f5-ba84-4d49-bffd-16c932df24c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.204364 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.204271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfsdm\" (UniqueName: \"kubernetes.io/projected/f56428f5-ba84-4d49-bffd-16c932df24c6-kube-api-access-pfsdm\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.204364 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.204358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f56428f5-ba84-4d49-bffd-16c932df24c6-maas-api-tls\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.204632 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.204410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f56428f5-ba84-4d49-bffd-16c932df24c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.205235 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.205210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:51.205316 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.205220    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f56428f5-ba84-4d49-bffd-16c932df24c6-openshift-service-ca\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.207031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.207009    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f56428f5-ba84-4d49-bffd-16c932df24c6-maas-api-tls\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.213398 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.213375    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfsdm\" (UniqueName: \"kubernetes.io/projected/f56428f5-ba84-4d49-bffd-16c932df24c6-kube-api-access-pfsdm\") pod \"maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.327159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.327117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:51.476796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.476771    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"]
Aug 03 16:33:51.478833 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:51.478801    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56428f5_ba84_4d49_bffd_16c932df24c6.slice/crio-e68762e3afe91fcd43f76ed783f872121fc9cc36e4b51caf73595464964b247c WatchSource:0}: Error finding container e68762e3afe91fcd43f76ed783f872121fc9cc36e4b51caf73595464964b247c: Status 404 returned error can't find the container with id e68762e3afe91fcd43f76ed783f872121fc9cc36e4b51caf73595464964b247c
Aug 03 16:33:51.554614 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.554581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8" event={"ID":"f56428f5-ba84-4d49-bffd-16c932df24c6","Type":"ContainerStarted","Data":"e68762e3afe91fcd43f76ed783f872121fc9cc36e4b51caf73595464964b247c"}
Aug 03 16:33:51.584753 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.584719    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:51.679138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.679101    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:33:51.698942 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.698899    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"]
Aug 03 16:33:51.703595 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.703568    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"]
Aug 03 16:33:51.703762 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.703671    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.706602 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.706579    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-51e95abc\""
Aug 03 16:33:51.707361 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.707343    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:51.707924 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.707907    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-51e95abc-dockercfg-2bvbf\""
Aug 03 16:33:51.724387 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.724351    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"]
Aug 03 16:33:51.727052 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.726985    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"]
Aug 03 16:33:51.809282 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.809244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c341038-6415-4c13-92bc-555d0edd426e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.809459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.809310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm8t8\" (UniqueName: \"kubernetes.io/projected/89093df6-a414-4996-b7a1-617109391bac-kube-api-access-xm8t8\") pod \"payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") " pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:51.809459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.809355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwzst\" (UniqueName: \"kubernetes.io/projected/6c341038-6415-4c13-92bc-555d0edd426e-kube-api-access-qwzst\") pod \"payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.809459 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.809389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89093df6-a414-4996-b7a1-617109391bac-plugins-config-volume\") pod \"payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") " pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:51.909907 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.909866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c341038-6415-4c13-92bc-555d0edd426e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.910068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.909940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm8t8\" (UniqueName: \"kubernetes.io/projected/89093df6-a414-4996-b7a1-617109391bac-kube-api-access-xm8t8\") pod \"payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") " pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:51.910068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.909992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwzst\" (UniqueName: \"kubernetes.io/projected/6c341038-6415-4c13-92bc-555d0edd426e-kube-api-access-qwzst\") pod \"payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.910068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.910039    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89093df6-a414-4996-b7a1-617109391bac-plugins-config-volume\") pod \"payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") " pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:51.910878 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.910853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89093df6-a414-4996-b7a1-617109391bac-plugins-config-volume\") pod \"payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") " pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:51.910916 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.910853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c341038-6415-4c13-92bc-555d0edd426e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.920760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.920722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwzst\" (UniqueName: \"kubernetes.io/projected/6c341038-6415-4c13-92bc-555d0edd426e-kube-api-access-qwzst\") pod \"payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:51.921021 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:51.921000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm8t8\" (UniqueName: \"kubernetes.io/projected/89093df6-a414-4996-b7a1-617109391bac-kube-api-access-xm8t8\") pod \"payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") " pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:52.016419 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.016384    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:52.022448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.022426    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:52.178928 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.178893    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"]
Aug 03 16:33:52.180228 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:52.180195    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89093df6_a414_4996_b7a1_617109391bac.slice/crio-9d3d87cf9aae86e662d7f13ea262a8d276564d467275c20f4387aca5ca32dcd6 WatchSource:0}: Error finding container 9d3d87cf9aae86e662d7f13ea262a8d276564d467275c20f4387aca5ca32dcd6: Status 404 returned error can't find the container with id 9d3d87cf9aae86e662d7f13ea262a8d276564d467275c20f4387aca5ca32dcd6
Aug 03 16:33:52.197875 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.197714    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"]
Aug 03 16:33:52.201296 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:33:52.201256    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c341038_6415_4c13_92bc_555d0edd426e.slice/crio-d885401ec2ed1c677ab1e060ad4f0c4d2ca10dd8bb21495dcc7abc8827affaf3 WatchSource:0}: Error finding container d885401ec2ed1c677ab1e060ad4f0c4d2ca10dd8bb21495dcc7abc8827affaf3: Status 404 returned error can't find the container with id d885401ec2ed1c677ab1e060ad4f0c4d2ca10dd8bb21495dcc7abc8827affaf3
Aug 03 16:33:52.562222 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.562178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8" event={"ID":"f56428f5-ba84-4d49-bffd-16c932df24c6","Type":"ContainerStarted","Data":"e1deca7cd1a25fd29d11a3d37327a46a5cc3183bf6d39dfcbba387a25b95dc7a"}
Aug 03 16:33:52.562453 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.562434    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:33:52.563890 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.563862    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq" event={"ID":"89093df6-a414-4996-b7a1-617109391bac","Type":"ContainerStarted","Data":"ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d"}
Aug 03 16:33:52.564011 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.563912    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq" event={"ID":"89093df6-a414-4996-b7a1-617109391bac","Type":"ContainerStarted","Data":"9d3d87cf9aae86e662d7f13ea262a8d276564d467275c20f4387aca5ca32dcd6"}
Aug 03 16:33:52.564011 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.563940    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:33:52.565382 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.565358    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn" event={"ID":"6c341038-6415-4c13-92bc-555d0edd426e","Type":"ContainerStarted","Data":"eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8"}
Aug 03 16:33:52.565511 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.565398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn" event={"ID":"6c341038-6415-4c13-92bc-555d0edd426e","Type":"ContainerStarted","Data":"d885401ec2ed1c677ab1e060ad4f0c4d2ca10dd8bb21495dcc7abc8827affaf3"}
Aug 03 16:33:52.565511 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.565428    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:33:52.581219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.581172    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8" podStartSLOduration=2.075413781 podStartE2EDuration="2.58115675s" totalImagesPullingTime="505.742969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:52.579146137 +0000 UTC m=+1792.128432465" watchObservedRunningTime="2026-08-03 16:33:52.58115675 +0000 UTC m=+1792.130443065"
Aug 03 16:33:52.595903 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.595846    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn" podStartSLOduration=1.595825158 podStartE2EDuration="1.595825158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:52.595663028 +0000 UTC m=+1792.144949369" watchObservedRunningTime="2026-08-03 16:33:52.595825158 +0000 UTC m=+1792.145111473"
Aug 03 16:33:52.614138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:52.614082    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq" podStartSLOduration=1.614065428 podStartE2EDuration="1.614065428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:52.612557446 +0000 UTC m=+1792.161843763" watchObservedRunningTime="2026-08-03 16:33:52.614065428 +0000 UTC m=+1792.163351744"
Aug 03 16:33:58.577447 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:33:58.577372    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:34:01.028267 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:01.028236    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:34:01.036346 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:01.036325    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:34:03.574237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:03.574172    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:34:03.574705 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:03.574522    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:34:15.950159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:15.950121    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"]
Aug 03 16:34:15.955711 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:15.954578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:15.957801 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:15.957775    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:34:15.962210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:15.962175    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"]
Aug 03 16:34:16.022019 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.021979    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/910674db-3cc7-46f1-8b02-626fbca48493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz\" (UID: \"910674db-3cc7-46f1-8b02-626fbca48493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:16.123458 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.123423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/910674db-3cc7-46f1-8b02-626fbca48493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz\" (UID: \"910674db-3cc7-46f1-8b02-626fbca48493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:16.124057 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.124037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/910674db-3cc7-46f1-8b02-626fbca48493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz\" (UID: \"910674db-3cc7-46f1-8b02-626fbca48493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:16.268946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.268869    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:16.406914 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.406888    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"]
Aug 03 16:34:16.409677 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:34:16.409646    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910674db_3cc7_46f1_8b02_626fbca48493.slice/crio-39ccad40a8894700c5b3f1560524325f2c2035d4a1ab66cdfb5c84c48381e418 WatchSource:0}: Error finding container 39ccad40a8894700c5b3f1560524325f2c2035d4a1ab66cdfb5c84c48381e418: Status 404 returned error can't find the container with id 39ccad40a8894700c5b3f1560524325f2c2035d4a1ab66cdfb5c84c48381e418
Aug 03 16:34:16.689230 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.689186    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz" event={"ID":"910674db-3cc7-46f1-8b02-626fbca48493","Type":"ContainerStarted","Data":"dba05c0766f6eec705e175cfca4750cc81c08c0bbfc000b48c9a0184d99dbc7f"}
Aug 03 16:34:16.689423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.689241    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz" event={"ID":"910674db-3cc7-46f1-8b02-626fbca48493","Type":"ContainerStarted","Data":"39ccad40a8894700c5b3f1560524325f2c2035d4a1ab66cdfb5c84c48381e418"}
Aug 03 16:34:16.705716 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:16.705670    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz" podStartSLOduration=1.705654011 podStartE2EDuration="1.705654011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:16.7055199 +0000 UTC m=+1816.254806225" watchObservedRunningTime="2026-08-03 16:34:16.705654011 +0000 UTC m=+1816.254940374"
Aug 03 16:34:17.695892 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:17.695854    2560 generic.go:356] "Generic (PLEG): container finished" podID="910674db-3cc7-46f1-8b02-626fbca48493" containerID="dba05c0766f6eec705e175cfca4750cc81c08c0bbfc000b48c9a0184d99dbc7f" exitCode=0
Aug 03 16:34:17.696293 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:17.695936    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz" event={"ID":"910674db-3cc7-46f1-8b02-626fbca48493","Type":"ContainerDied","Data":"dba05c0766f6eec705e175cfca4750cc81c08c0bbfc000b48c9a0184d99dbc7f"}
Aug 03 16:34:18.841373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:18.841344    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:18.952413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:18.952343    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/910674db-3cc7-46f1-8b02-626fbca48493-maas-api-service-ca\") pod \"910674db-3cc7-46f1-8b02-626fbca48493\" (UID: \"910674db-3cc7-46f1-8b02-626fbca48493\") "
Aug 03 16:34:18.952700 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:18.952678    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/910674db-3cc7-46f1-8b02-626fbca48493-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "910674db-3cc7-46f1-8b02-626fbca48493" (UID: "910674db-3cc7-46f1-8b02-626fbca48493"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:19.053908 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:19.053881    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/910674db-3cc7-46f1-8b02-626fbca48493-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:19.707273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:19.707246    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"
Aug 03 16:34:19.707273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:19.707253    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz" event={"ID":"910674db-3cc7-46f1-8b02-626fbca48493","Type":"ContainerDied","Data":"39ccad40a8894700c5b3f1560524325f2c2035d4a1ab66cdfb5c84c48381e418"}
Aug 03 16:34:19.707565 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:19.707304    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39ccad40a8894700c5b3f1560524325f2c2035d4a1ab66cdfb5c84c48381e418"
Aug 03 16:34:21.572818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.572786    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"]
Aug 03 16:34:21.573315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.573004    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8" podUID="f56428f5-ba84-4d49-bffd-16c932df24c6" containerName="maas-api" containerID="cri-o://e1deca7cd1a25fd29d11a3d37327a46a5cc3183bf6d39dfcbba387a25b95dc7a" gracePeriod=30
Aug 03 16:34:21.613028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.612996    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:21.651301 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.651266    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"]
Aug 03 16:34:21.651569 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.651502    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq" podUID="89093df6-a414-4996-b7a1-617109391bac" containerName="payload-processing" containerID="cri-o://ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d" gracePeriod=30
Aug 03 16:34:21.661491 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.661463    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"]
Aug 03 16:34:21.661863 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.661836    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn" podUID="6c341038-6415-4c13-92bc-555d0edd426e" containerName="payload-pre-processing" containerID="cri-o://eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8" gracePeriod=30
Aug 03 16:34:21.711979 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.711949    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:21.719016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.718982    2560 generic.go:356] "Generic (PLEG): container finished" podID="f56428f5-ba84-4d49-bffd-16c932df24c6" containerID="e1deca7cd1a25fd29d11a3d37327a46a5cc3183bf6d39dfcbba387a25b95dc7a" exitCode=0
Aug 03 16:34:21.719182 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.719056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8" event={"ID":"f56428f5-ba84-4d49-bffd-16c932df24c6","Type":"ContainerDied","Data":"e1deca7cd1a25fd29d11a3d37327a46a5cc3183bf6d39dfcbba387a25b95dc7a"}
Aug 03 16:34:21.802235 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.801468    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"]
Aug 03 16:34:21.804896 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.804818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:21.805061 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.805023    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51e95abc-cs2fz"]
Aug 03 16:34:21.869874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.869791    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:21.962662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.962623    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:34:21.967074 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.967055    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:34:21.971853 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.971825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:21.993463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:21.993443    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:34:22.083126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083044    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f56428f5-ba84-4d49-bffd-16c932df24c6-openshift-service-ca\") pod \"f56428f5-ba84-4d49-bffd-16c932df24c6\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") "
Aug 03 16:34:22.083126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083090    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89093df6-a414-4996-b7a1-617109391bac-plugins-config-volume\") pod \"89093df6-a414-4996-b7a1-617109391bac\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") "
Aug 03 16:34:22.083126 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083118    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c341038-6415-4c13-92bc-555d0edd426e-plugins-config-volume\") pod \"6c341038-6415-4c13-92bc-555d0edd426e\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") "
Aug 03 16:34:22.083423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083140    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm8t8\" (UniqueName: \"kubernetes.io/projected/89093df6-a414-4996-b7a1-617109391bac-kube-api-access-xm8t8\") pod \"89093df6-a414-4996-b7a1-617109391bac\" (UID: \"89093df6-a414-4996-b7a1-617109391bac\") "
Aug 03 16:34:22.083423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083290    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f56428f5-ba84-4d49-bffd-16c932df24c6-maas-api-tls\") pod \"f56428f5-ba84-4d49-bffd-16c932df24c6\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") "
Aug 03 16:34:22.083423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083380    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwzst\" (UniqueName: \"kubernetes.io/projected/6c341038-6415-4c13-92bc-555d0edd426e-kube-api-access-qwzst\") pod \"6c341038-6415-4c13-92bc-555d0edd426e\" (UID: \"6c341038-6415-4c13-92bc-555d0edd426e\") "
Aug 03 16:34:22.083423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083416    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfsdm\" (UniqueName: \"kubernetes.io/projected/f56428f5-ba84-4d49-bffd-16c932df24c6-kube-api-access-pfsdm\") pod \"f56428f5-ba84-4d49-bffd-16c932df24c6\" (UID: \"f56428f5-ba84-4d49-bffd-16c932df24c6\") "
Aug 03 16:34:22.083653 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083447    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56428f5-ba84-4d49-bffd-16c932df24c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f56428f5-ba84-4d49-bffd-16c932df24c6" (UID: "f56428f5-ba84-4d49-bffd-16c932df24c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:22.083653 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083506    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89093df6-a414-4996-b7a1-617109391bac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89093df6-a414-4996-b7a1-617109391bac" (UID: "89093df6-a414-4996-b7a1-617109391bac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:22.083653 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.083511    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c341038-6415-4c13-92bc-555d0edd426e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c341038-6415-4c13-92bc-555d0edd426e" (UID: "6c341038-6415-4c13-92bc-555d0edd426e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:22.084263 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.084007    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89093df6-a414-4996-b7a1-617109391bac-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.084263 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.084032    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c341038-6415-4c13-92bc-555d0edd426e-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.084263 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.084046    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f56428f5-ba84-4d49-bffd-16c932df24c6-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.085999 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.085962    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f56428f5-ba84-4d49-bffd-16c932df24c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f56428f5-ba84-4d49-bffd-16c932df24c6" (UID: "f56428f5-ba84-4d49-bffd-16c932df24c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:34:22.085999 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.085975    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c341038-6415-4c13-92bc-555d0edd426e-kube-api-access-qwzst" (OuterVolumeSpecName: "kube-api-access-qwzst") pod "6c341038-6415-4c13-92bc-555d0edd426e" (UID: "6c341038-6415-4c13-92bc-555d0edd426e"). InnerVolumeSpecName "kube-api-access-qwzst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:34:22.086146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.086097    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89093df6-a414-4996-b7a1-617109391bac-kube-api-access-xm8t8" (OuterVolumeSpecName: "kube-api-access-xm8t8") pod "89093df6-a414-4996-b7a1-617109391bac" (UID: "89093df6-a414-4996-b7a1-617109391bac"). InnerVolumeSpecName "kube-api-access-xm8t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:34:22.086767 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.086751    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56428f5-ba84-4d49-bffd-16c932df24c6-kube-api-access-pfsdm" (OuterVolumeSpecName: "kube-api-access-pfsdm") pod "f56428f5-ba84-4d49-bffd-16c932df24c6" (UID: "f56428f5-ba84-4d49-bffd-16c932df24c6"). InnerVolumeSpecName "kube-api-access-pfsdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:34:22.184843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.184804    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm8t8\" (UniqueName: \"kubernetes.io/projected/89093df6-a414-4996-b7a1-617109391bac-kube-api-access-xm8t8\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.184843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.184837    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f56428f5-ba84-4d49-bffd-16c932df24c6-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.184843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.184846    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwzst\" (UniqueName: \"kubernetes.io/projected/6c341038-6415-4c13-92bc-555d0edd426e-kube-api-access-qwzst\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.184843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.184855    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfsdm\" (UniqueName: \"kubernetes.io/projected/f56428f5-ba84-4d49-bffd-16c932df24c6-kube-api-access-pfsdm\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:22.189612 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.189577    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:22.409976 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.409876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:22.722408 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.722321    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:22.725855 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.725827    2560 generic.go:356] "Generic (PLEG): container finished" podID="89093df6-a414-4996-b7a1-617109391bac" containerID="ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d" exitCode=0
Aug 03 16:34:22.725992 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.725878    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"
Aug 03 16:34:22.725992 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.725915    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq" event={"ID":"89093df6-a414-4996-b7a1-617109391bac","Type":"ContainerDied","Data":"ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d"}
Aug 03 16:34:22.725992 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.725976    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq" event={"ID":"89093df6-a414-4996-b7a1-617109391bac","Type":"ContainerDied","Data":"9d3d87cf9aae86e662d7f13ea262a8d276564d467275c20f4387aca5ca32dcd6"}
Aug 03 16:34:22.726122 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.726009    2560 scope.go:117] "RemoveContainer" containerID="ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d"
Aug 03 16:34:22.727406 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.727347    2560 generic.go:356] "Generic (PLEG): container finished" podID="6c341038-6415-4c13-92bc-555d0edd426e" containerID="eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8" exitCode=0
Aug 03 16:34:22.727610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.727506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn" event={"ID":"6c341038-6415-4c13-92bc-555d0edd426e","Type":"ContainerDied","Data":"eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8"}
Aug 03 16:34:22.727610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.727582    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn" event={"ID":"6c341038-6415-4c13-92bc-555d0edd426e","Type":"ContainerDied","Data":"d885401ec2ed1c677ab1e060ad4f0c4d2ca10dd8bb21495dcc7abc8827affaf3"}
Aug 03 16:34:22.727610 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.727513    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"
Aug 03 16:34:22.733594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.733564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8" event={"ID":"f56428f5-ba84-4d49-bffd-16c932df24c6","Type":"ContainerDied","Data":"e68762e3afe91fcd43f76ed783f872121fc9cc36e4b51caf73595464964b247c"}
Aug 03 16:34:22.733715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.733571    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"
Aug 03 16:34:22.737059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.736992    2560 scope.go:117] "RemoveContainer" containerID="ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d"
Aug 03 16:34:22.737308 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:34:22.737279    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d\": container with ID starting with ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d not found: ID does not exist" containerID="ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d"
Aug 03 16:34:22.737441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.737314    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d"} err="failed to get container status \"ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d\": rpc error: code = NotFound desc = could not find container \"ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d\": container with ID starting with ff9c11f7e04704512951ab7ccd4d27efc2bb8f52c1e037f748dea53d04a3b39d not found: ID does not exist"
Aug 03 16:34:22.737441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.737352    2560 scope.go:117] "RemoveContainer" containerID="eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8"
Aug 03 16:34:22.748503 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.748483    2560 scope.go:117] "RemoveContainer" containerID="eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8"
Aug 03 16:34:22.748837 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:34:22.748816    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8\": container with ID starting with eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8 not found: ID does not exist" containerID="eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8"
Aug 03 16:34:22.748900 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.748849    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8"} err="failed to get container status \"eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8\": rpc error: code = NotFound desc = could not find container \"eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8\": container with ID starting with eb66316e5e6598788435a6321cba81ef8feaeea0aabe6a796faa0814f54404d8 not found: ID does not exist"
Aug 03 16:34:22.748900 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.748871    2560 scope.go:117] "RemoveContainer" containerID="e1deca7cd1a25fd29d11a3d37327a46a5cc3183bf6d39dfcbba387a25b95dc7a"
Aug 03 16:34:22.757107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.757083    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"]
Aug 03 16:34:22.761003 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.760982    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51e95abc-5f4c4b8657-tb4kq"]
Aug 03 16:34:22.771028 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.771007    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"]
Aug 03 16:34:22.775682 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.775655    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-51e95abc-7bddc69cd4-zkjfn"]
Aug 03 16:34:22.785708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.785685    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"]
Aug 03 16:34:22.791216 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.791188    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51e95abc-6dc4b557cf-2l9d8"]
Aug 03 16:34:22.974613 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.974513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:22.987207 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.987178    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c341038-6415-4c13-92bc-555d0edd426e" path="/var/lib/kubelet/pods/6c341038-6415-4c13-92bc-555d0edd426e/volumes"
Aug 03 16:34:22.987568 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.987541    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89093df6-a414-4996-b7a1-617109391bac" path="/var/lib/kubelet/pods/89093df6-a414-4996-b7a1-617109391bac/volumes"
Aug 03 16:34:22.987824 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.987812    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="910674db-3cc7-46f1-8b02-626fbca48493" path="/var/lib/kubelet/pods/910674db-3cc7-46f1-8b02-626fbca48493/volumes"
Aug 03 16:34:22.988148 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:22.988128    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f56428f5-ba84-4d49-bffd-16c932df24c6" path="/var/lib/kubelet/pods/f56428f5-ba84-4d49-bffd-16c932df24c6/volumes"
Aug 03 16:34:25.036652 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:25.036617    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:25.133710 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:25.133677    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:26.123316 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123274    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"]
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123718    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89093df6-a414-4996-b7a1-617109391bac" containerName="payload-processing"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123732    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89093df6-a414-4996-b7a1-617109391bac" containerName="payload-processing"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123740    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910674db-3cc7-46f1-8b02-626fbca48493" containerName="revoke-keys"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123749    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="910674db-3cc7-46f1-8b02-626fbca48493" containerName="revoke-keys"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123772    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c341038-6415-4c13-92bc-555d0edd426e" containerName="payload-pre-processing"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123778    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c341038-6415-4c13-92bc-555d0edd426e" containerName="payload-pre-processing"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123789    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56428f5-ba84-4d49-bffd-16c932df24c6" containerName="maas-api"
Aug 03 16:34:26.123794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123794    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56428f5-ba84-4d49-bffd-16c932df24c6" containerName="maas-api"
Aug 03 16:34:26.124198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123853    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56428f5-ba84-4d49-bffd-16c932df24c6" containerName="maas-api"
Aug 03 16:34:26.124198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123862    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89093df6-a414-4996-b7a1-617109391bac" containerName="payload-processing"
Aug 03 16:34:26.124198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123871    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c341038-6415-4c13-92bc-555d0edd426e" containerName="payload-pre-processing"
Aug 03 16:34:26.124198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.123877    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="910674db-3cc7-46f1-8b02-626fbca48493" containerName="revoke-keys"
Aug 03 16:34:26.128626 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.128604    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.131335 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.131309    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4f378709-serving-cert\""
Aug 03 16:34:26.139689 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.139663    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"]
Aug 03 16:34:26.156775 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.156736    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:26.224215 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.224166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.224366 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.224219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc380932-6576-4d70-98df-b1625d8abf6a-openshift-service-ca\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.224431 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.224388    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px8mv\" (UniqueName: \"kubernetes.io/projected/cc380932-6576-4d70-98df-b1625d8abf6a-kube-api-access-px8mv\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.325068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.325035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px8mv\" (UniqueName: \"kubernetes.io/projected/cc380932-6576-4d70-98df-b1625d8abf6a-kube-api-access-px8mv\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.325246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.325079    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.325246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.325100    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc380932-6576-4d70-98df-b1625d8abf6a-openshift-service-ca\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.325246 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:34:26.325200    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-serving-cert: secret "maas-api-e2e-ait-4f378709-serving-cert" not found
Aug 03 16:34:26.325362 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:34:26.325283    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls podName:cc380932-6576-4d70-98df-b1625d8abf6a nodeName:}" failed. No retries permitted until 2026-08-03 16:34:26.825266002 +0000 UTC m=+1826.374552296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls") pod "maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" (UID: "cc380932-6576-4d70-98df-b1625d8abf6a") : secret "maas-api-e2e-ait-4f378709-serving-cert" not found
Aug 03 16:34:26.325743 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.325724    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc380932-6576-4d70-98df-b1625d8abf6a-openshift-service-ca\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.334471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.334440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px8mv\" (UniqueName: \"kubernetes.io/projected/cc380932-6576-4d70-98df-b1625d8abf6a-kube-api-access-px8mv\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.589484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.589450    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:26.692966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.692930    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:26.768696 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.768662    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:26.772902 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.772877    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"]
Aug 03 16:34:26.776402 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.776385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:26.779032 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.779010    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4f378709-dockercfg-n2zdl\""
Aug 03 16:34:26.779032 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.779027    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4f378709\""
Aug 03 16:34:26.787416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.787394    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"]
Aug 03 16:34:26.823732 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.823704    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"]
Aug 03 16:34:26.828055 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.828026    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:26.831023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.830986    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.832582 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.832556    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"]
Aug 03 16:34:26.834409 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.834376    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls\") pod \"maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:26.931811 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.931779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9rt\" (UniqueName: \"kubernetes.io/projected/f9eb0025-3014-4a2e-9b03-3333ae461d2d-kube-api-access-5l9rt\") pod \"payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") " pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:26.931983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.931840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8zb\" (UniqueName: \"kubernetes.io/projected/c9e1b405-6e16-471f-8b8b-12f5847bcba2-kube-api-access-cg8zb\") pod \"payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:26.931983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.931938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e1b405-6e16-471f-8b8b-12f5847bcba2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:26.932117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:26.931986    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9eb0025-3014-4a2e-9b03-3333ae461d2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") " pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.033268 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.033230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9rt\" (UniqueName: \"kubernetes.io/projected/f9eb0025-3014-4a2e-9b03-3333ae461d2d-kube-api-access-5l9rt\") pod \"payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") " pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.033453 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.033304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8zb\" (UniqueName: \"kubernetes.io/projected/c9e1b405-6e16-471f-8b8b-12f5847bcba2-kube-api-access-cg8zb\") pod \"payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:27.033453 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.033357    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e1b405-6e16-471f-8b8b-12f5847bcba2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:27.033453 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.033395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9eb0025-3014-4a2e-9b03-3333ae461d2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") " pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.034090 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.034065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e1b405-6e16-471f-8b8b-12f5847bcba2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:27.034192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.034128    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9eb0025-3014-4a2e-9b03-3333ae461d2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") " pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.042401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.042380    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:27.042555 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.042406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9rt\" (UniqueName: \"kubernetes.io/projected/f9eb0025-3014-4a2e-9b03-3333ae461d2d-kube-api-access-5l9rt\") pod \"payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") " pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.042762 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.042745    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8zb\" (UniqueName: \"kubernetes.io/projected/c9e1b405-6e16-471f-8b8b-12f5847bcba2-kube-api-access-cg8zb\") pod \"payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:27.087355 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.087328    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:27.155407 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.155166    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.194361 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.194265    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"]
Aug 03 16:34:27.267694 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.267661    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"]
Aug 03 16:34:27.270406 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:34:27.270372    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e1b405_6e16_471f_8b8b_12f5847bcba2.slice/crio-1207bf8c93779b3ec953963e606920f39764d6e6f2f88d7a91aad3182dd456e8 WatchSource:0}: Error finding container 1207bf8c93779b3ec953963e606920f39764d6e6f2f88d7a91aad3182dd456e8: Status 404 returned error can't find the container with id 1207bf8c93779b3ec953963e606920f39764d6e6f2f88d7a91aad3182dd456e8
Aug 03 16:34:27.309687 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.309664    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"]
Aug 03 16:34:27.314990 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:34:27.314965    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9eb0025_3014_4a2e_9b03_3333ae461d2d.slice/crio-4eed22ebd2b34650a2b95efdf87ecaf8907e93b3690d5cd8f60b09de52e32999 WatchSource:0}: Error finding container 4eed22ebd2b34650a2b95efdf87ecaf8907e93b3690d5cd8f60b09de52e32999: Status 404 returned error can't find the container with id 4eed22ebd2b34650a2b95efdf87ecaf8907e93b3690d5cd8f60b09de52e32999
Aug 03 16:34:27.768447 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.768408    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv" event={"ID":"f9eb0025-3014-4a2e-9b03-3333ae461d2d","Type":"ContainerStarted","Data":"558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d"}
Aug 03 16:34:27.768672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.768487    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv" event={"ID":"f9eb0025-3014-4a2e-9b03-3333ae461d2d","Type":"ContainerStarted","Data":"4eed22ebd2b34650a2b95efdf87ecaf8907e93b3690d5cd8f60b09de52e32999"}
Aug 03 16:34:27.768672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.768516    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:27.769980 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.769948    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" event={"ID":"cc380932-6576-4d70-98df-b1625d8abf6a","Type":"ContainerStarted","Data":"4ecfdad5ab0a9a61e03341d2a4fef66ff34246ddf3806fbb6e9df20d38af0d3f"}
Aug 03 16:34:27.770104 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.770007    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" event={"ID":"cc380932-6576-4d70-98df-b1625d8abf6a","Type":"ContainerStarted","Data":"e43594a856ef816b35d1862b9fb9ced7ac441797b2e4c1243b238f3599108a92"}
Aug 03 16:34:27.770104 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.770038    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:27.771455 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.771430    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr" event={"ID":"c9e1b405-6e16-471f-8b8b-12f5847bcba2","Type":"ContainerStarted","Data":"0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60"}
Aug 03 16:34:27.771561 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.771469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr" event={"ID":"c9e1b405-6e16-471f-8b8b-12f5847bcba2","Type":"ContainerStarted","Data":"1207bf8c93779b3ec953963e606920f39764d6e6f2f88d7a91aad3182dd456e8"}
Aug 03 16:34:27.771561 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.771502    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:27.787252 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.787209    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv" podStartSLOduration=1.7871972600000001 podStartE2EDuration="1.78719726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:27.785558074 +0000 UTC m=+1827.334844391" watchObservedRunningTime="2026-08-03 16:34:27.78719726 +0000 UTC m=+1827.336483575"
Aug 03 16:34:27.804178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.804127    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr" podStartSLOduration=1.8041090629999998 podStartE2EDuration="1.804109063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:27.801479088 +0000 UTC m=+1827.350765397" watchObservedRunningTime="2026-08-03 16:34:27.804109063 +0000 UTC m=+1827.353395382"
Aug 03 16:34:27.820273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:27.820215    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" podStartSLOduration=1.371836657 podStartE2EDuration="1.820194772s" totalImagesPullingTime="448.358115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:27.818567404 +0000 UTC m=+1827.367853717" watchObservedRunningTime="2026-08-03 16:34:27.820194772 +0000 UTC m=+1827.369481088"
Aug 03 16:34:33.782016 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:33.781989    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:38.779098 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:38.779069    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:38.779621 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:38.779365    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:50.876233 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:50.876200    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"]
Aug 03 16:34:50.879581 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:50.879562    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:50.882778 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:50.882755    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:34:50.888715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:50.888638    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"]
Aug 03 16:34:50.932470 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:50.932435    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37efec57-39b8-45e4-a5be-e1892e753dd1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f378709-qhtmp\" (UID: \"37efec57-39b8-45e4-a5be-e1892e753dd1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:51.033079 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.033047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37efec57-39b8-45e4-a5be-e1892e753dd1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f378709-qhtmp\" (UID: \"37efec57-39b8-45e4-a5be-e1892e753dd1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:51.033776 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.033749    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37efec57-39b8-45e4-a5be-e1892e753dd1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f378709-qhtmp\" (UID: \"37efec57-39b8-45e4-a5be-e1892e753dd1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:51.192636 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.192517    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:51.328910 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.328887    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"]
Aug 03 16:34:51.330965 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:34:51.330939    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37efec57_39b8_45e4_a5be_e1892e753dd1.slice/crio-1923930a0309ed6050d27c00aed7e2e30bf61141e7ae14db97787b75f6578f9c WatchSource:0}: Error finding container 1923930a0309ed6050d27c00aed7e2e30bf61141e7ae14db97787b75f6578f9c: Status 404 returned error can't find the container with id 1923930a0309ed6050d27c00aed7e2e30bf61141e7ae14db97787b75f6578f9c
Aug 03 16:34:51.904332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.904288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp" event={"ID":"37efec57-39b8-45e4-a5be-e1892e753dd1","Type":"ContainerStarted","Data":"6f2098976d3048abd1fead580fab0936416f84946a856ef0cb3ed47d09a6a297"}
Aug 03 16:34:51.904777 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.904362    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp" event={"ID":"37efec57-39b8-45e4-a5be-e1892e753dd1","Type":"ContainerStarted","Data":"1923930a0309ed6050d27c00aed7e2e30bf61141e7ae14db97787b75f6578f9c"}
Aug 03 16:34:51.921757 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:51.921708    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp" podStartSLOduration=1.921691335 podStartE2EDuration="1.921691335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:51.920577102 +0000 UTC m=+1851.469863419" watchObservedRunningTime="2026-08-03 16:34:51.921691335 +0000 UTC m=+1851.470977650"
Aug 03 16:34:52.910871 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:52.910838    2560 generic.go:356] "Generic (PLEG): container finished" podID="37efec57-39b8-45e4-a5be-e1892e753dd1" containerID="6f2098976d3048abd1fead580fab0936416f84946a856ef0cb3ed47d09a6a297" exitCode=0
Aug 03 16:34:52.911248 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:52.910920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp" event={"ID":"37efec57-39b8-45e4-a5be-e1892e753dd1","Type":"ContainerDied","Data":"6f2098976d3048abd1fead580fab0936416f84946a856ef0cb3ed47d09a6a297"}
Aug 03 16:34:54.039471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.039446    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:54.161149 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.161117    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37efec57-39b8-45e4-a5be-e1892e753dd1-maas-api-service-ca\") pod \"37efec57-39b8-45e4-a5be-e1892e753dd1\" (UID: \"37efec57-39b8-45e4-a5be-e1892e753dd1\") "
Aug 03 16:34:54.161494 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.161472    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37efec57-39b8-45e4-a5be-e1892e753dd1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37efec57-39b8-45e4-a5be-e1892e753dd1" (UID: "37efec57-39b8-45e4-a5be-e1892e753dd1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:54.262088 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.262004    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37efec57-39b8-45e4-a5be-e1892e753dd1-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:54.922697 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.922661    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp" event={"ID":"37efec57-39b8-45e4-a5be-e1892e753dd1","Type":"ContainerDied","Data":"1923930a0309ed6050d27c00aed7e2e30bf61141e7ae14db97787b75f6578f9c"}
Aug 03 16:34:54.922697 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.922695    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"
Aug 03 16:34:54.922927 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:54.922712    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1923930a0309ed6050d27c00aed7e2e30bf61141e7ae14db97787b75f6578f9c"
Aug 03 16:34:56.766089 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.766049    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"]
Aug 03 16:34:56.766613 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.766307    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" podUID="cc380932-6576-4d70-98df-b1625d8abf6a" containerName="maas-api" containerID="cri-o://4ecfdad5ab0a9a61e03341d2a4fef66ff34246ddf3806fbb6e9df20d38af0d3f" gracePeriod=30
Aug 03 16:34:56.800756 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.800727    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:56.827632 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.827598    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"]
Aug 03 16:34:56.827882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.827855    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv" podUID="f9eb0025-3014-4a2e-9b03-3333ae461d2d" containerName="payload-processing" containerID="cri-o://558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d" gracePeriod=30
Aug 03 16:34:56.837018 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.836987    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"]
Aug 03 16:34:56.837259 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.837234    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr" podUID="c9e1b405-6e16-471f-8b8b-12f5847bcba2" containerName="payload-pre-processing" containerID="cri-o://0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60" gracePeriod=30
Aug 03 16:34:56.902796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.902753    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:56.934614 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.934587    2560 generic.go:356] "Generic (PLEG): container finished" podID="cc380932-6576-4d70-98df-b1625d8abf6a" containerID="4ecfdad5ab0a9a61e03341d2a4fef66ff34246ddf3806fbb6e9df20d38af0d3f" exitCode=0
Aug 03 16:34:56.934794 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.934672    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" event={"ID":"cc380932-6576-4d70-98df-b1625d8abf6a","Type":"ContainerDied","Data":"4ecfdad5ab0a9a61e03341d2a4fef66ff34246ddf3806fbb6e9df20d38af0d3f"}
Aug 03 16:34:56.990739 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.990614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:56.992709 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.992677    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"]
Aug 03 16:34:56.992858 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:56.992722    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f378709-qhtmp"]
Aug 03 16:34:57.057128 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.057047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:57.138373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.138339    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:57.142381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.142357    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:57.159212 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.159180    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:57.189282 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.189253    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls\") pod \"cc380932-6576-4d70-98df-b1625d8abf6a\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") "
Aug 03 16:34:57.189445 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.189293    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc380932-6576-4d70-98df-b1625d8abf6a-openshift-service-ca\") pod \"cc380932-6576-4d70-98df-b1625d8abf6a\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") "
Aug 03 16:34:57.189445 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.189316    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px8mv\" (UniqueName: \"kubernetes.io/projected/cc380932-6576-4d70-98df-b1625d8abf6a-kube-api-access-px8mv\") pod \"cc380932-6576-4d70-98df-b1625d8abf6a\" (UID: \"cc380932-6576-4d70-98df-b1625d8abf6a\") "
Aug 03 16:34:57.189736 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.189704    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc380932-6576-4d70-98df-b1625d8abf6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc380932-6576-4d70-98df-b1625d8abf6a" (UID: "cc380932-6576-4d70-98df-b1625d8abf6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:57.191477 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.191447    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc380932-6576-4d70-98df-b1625d8abf6a-kube-api-access-px8mv" (OuterVolumeSpecName: "kube-api-access-px8mv") pod "cc380932-6576-4d70-98df-b1625d8abf6a" (UID: "cc380932-6576-4d70-98df-b1625d8abf6a"). InnerVolumeSpecName "kube-api-access-px8mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:34:57.191604 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.191487    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc380932-6576-4d70-98df-b1625d8abf6a" (UID: "cc380932-6576-4d70-98df-b1625d8abf6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:34:57.289904 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.289868    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9eb0025-3014-4a2e-9b03-3333ae461d2d-plugins-config-volume\") pod \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") "
Aug 03 16:34:57.290104 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.289929    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e1b405-6e16-471f-8b8b-12f5847bcba2-plugins-config-volume\") pod \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") "
Aug 03 16:34:57.290104 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.289987    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l9rt\" (UniqueName: \"kubernetes.io/projected/f9eb0025-3014-4a2e-9b03-3333ae461d2d-kube-api-access-5l9rt\") pod \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\" (UID: \"f9eb0025-3014-4a2e-9b03-3333ae461d2d\") "
Aug 03 16:34:57.290104 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.290019    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg8zb\" (UniqueName: \"kubernetes.io/projected/c9e1b405-6e16-471f-8b8b-12f5847bcba2-kube-api-access-cg8zb\") pod \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\" (UID: \"c9e1b405-6e16-471f-8b8b-12f5847bcba2\") "
Aug 03 16:34:57.290357 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.290297    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc380932-6576-4d70-98df-b1625d8abf6a-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.290357 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.290329    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc380932-6576-4d70-98df-b1625d8abf6a-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.290357 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.290345    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px8mv\" (UniqueName: \"kubernetes.io/projected/cc380932-6576-4d70-98df-b1625d8abf6a-kube-api-access-px8mv\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.290357 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.290292    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9eb0025-3014-4a2e-9b03-3333ae461d2d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9eb0025-3014-4a2e-9b03-3333ae461d2d" (UID: "f9eb0025-3014-4a2e-9b03-3333ae461d2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:57.290514 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.290367    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e1b405-6e16-471f-8b8b-12f5847bcba2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9e1b405-6e16-471f-8b8b-12f5847bcba2" (UID: "c9e1b405-6e16-471f-8b8b-12f5847bcba2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:34:57.292174 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.292156    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9e1b405-6e16-471f-8b8b-12f5847bcba2-kube-api-access-cg8zb" (OuterVolumeSpecName: "kube-api-access-cg8zb") pod "c9e1b405-6e16-471f-8b8b-12f5847bcba2" (UID: "c9e1b405-6e16-471f-8b8b-12f5847bcba2"). InnerVolumeSpecName "kube-api-access-cg8zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:34:57.292252 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.292181    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9eb0025-3014-4a2e-9b03-3333ae461d2d-kube-api-access-5l9rt" (OuterVolumeSpecName: "kube-api-access-5l9rt") pod "f9eb0025-3014-4a2e-9b03-3333ae461d2d" (UID: "f9eb0025-3014-4a2e-9b03-3333ae461d2d"). InnerVolumeSpecName "kube-api-access-5l9rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:34:57.391455 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.391366    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l9rt\" (UniqueName: \"kubernetes.io/projected/f9eb0025-3014-4a2e-9b03-3333ae461d2d-kube-api-access-5l9rt\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.391455 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.391399    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg8zb\" (UniqueName: \"kubernetes.io/projected/c9e1b405-6e16-471f-8b8b-12f5847bcba2-kube-api-access-cg8zb\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.391455 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.391408    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9eb0025-3014-4a2e-9b03-3333ae461d2d-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.391455 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.391417    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9e1b405-6e16-471f-8b8b-12f5847bcba2-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:34:57.941310 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.941269    2560 generic.go:356] "Generic (PLEG): container finished" podID="f9eb0025-3014-4a2e-9b03-3333ae461d2d" containerID="558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d" exitCode=0
Aug 03 16:34:57.941781 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.941315    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"
Aug 03 16:34:57.941781 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.941358    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv" event={"ID":"f9eb0025-3014-4a2e-9b03-3333ae461d2d","Type":"ContainerDied","Data":"558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d"}
Aug 03 16:34:57.941781 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.941414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv" event={"ID":"f9eb0025-3014-4a2e-9b03-3333ae461d2d","Type":"ContainerDied","Data":"4eed22ebd2b34650a2b95efdf87ecaf8907e93b3690d5cd8f60b09de52e32999"}
Aug 03 16:34:57.941781 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.941441    2560 scope.go:117] "RemoveContainer" containerID="558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d"
Aug 03 16:34:57.942890 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.942866    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"
Aug 03 16:34:57.943014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.942894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f" event={"ID":"cc380932-6576-4d70-98df-b1625d8abf6a","Type":"ContainerDied","Data":"e43594a856ef816b35d1862b9fb9ced7ac441797b2e4c1243b238f3599108a92"}
Aug 03 16:34:57.944192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.944168    2560 generic.go:356] "Generic (PLEG): container finished" podID="c9e1b405-6e16-471f-8b8b-12f5847bcba2" containerID="0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60" exitCode=0
Aug 03 16:34:57.944302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.944200    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr" event={"ID":"c9e1b405-6e16-471f-8b8b-12f5847bcba2","Type":"ContainerDied","Data":"0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60"}
Aug 03 16:34:57.944302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.944223    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"
Aug 03 16:34:57.944302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.944229    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr" event={"ID":"c9e1b405-6e16-471f-8b8b-12f5847bcba2","Type":"ContainerDied","Data":"1207bf8c93779b3ec953963e606920f39764d6e6f2f88d7a91aad3182dd456e8"}
Aug 03 16:34:57.951720 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.951698    2560 scope.go:117] "RemoveContainer" containerID="558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d"
Aug 03 16:34:57.952032 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:34:57.952007    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d\": container with ID starting with 558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d not found: ID does not exist" containerID="558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d"
Aug 03 16:34:57.952117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.952042    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d"} err="failed to get container status \"558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d\": rpc error: code = NotFound desc = could not find container \"558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d\": container with ID starting with 558967158cc64752490cc918168858b9ecbc44c78ed468defb7cbeefc974929d not found: ID does not exist"
Aug 03 16:34:57.952117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.952066    2560 scope.go:117] "RemoveContainer" containerID="4ecfdad5ab0a9a61e03341d2a4fef66ff34246ddf3806fbb6e9df20d38af0d3f"
Aug 03 16:34:57.961021 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.961004    2560 scope.go:117] "RemoveContainer" containerID="0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60"
Aug 03 16:34:57.969023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.969002    2560 scope.go:117] "RemoveContainer" containerID="0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60"
Aug 03 16:34:57.969312 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:34:57.969291    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60\": container with ID starting with 0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60 not found: ID does not exist" containerID="0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60"
Aug 03 16:34:57.969368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.969320    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60"} err="failed to get container status \"0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60\": rpc error: code = NotFound desc = could not find container \"0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60\": container with ID starting with 0f635ffb34b2ac7d28d6c2fb0bdf9197cb2bfc20e16c146e4c777af5d43bed60 not found: ID does not exist"
Aug 03 16:34:57.974370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.974347    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"]
Aug 03 16:34:57.981667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.981646    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4f378709-68dfcdb8d-8xccv"]
Aug 03 16:34:57.993745 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:57.993720    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"]
Aug 03 16:34:58.001137 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.001117    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4f378709-58b59d99bd-l5ntr"]
Aug 03 16:34:58.013313 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.013292    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"]
Aug 03 16:34:58.018636 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.018619    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4f378709-6b4d8f9877-qjv7f"]
Aug 03 16:34:58.193558 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.193495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:58.419416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.419380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:58.646882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.646848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:58.946905 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.946817    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:34:58.987350 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.987310    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37efec57-39b8-45e4-a5be-e1892e753dd1" path="/var/lib/kubelet/pods/37efec57-39b8-45e4-a5be-e1892e753dd1/volumes"
Aug 03 16:34:58.987876 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.987857    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9e1b405-6e16-471f-8b8b-12f5847bcba2" path="/var/lib/kubelet/pods/c9e1b405-6e16-471f-8b8b-12f5847bcba2/volumes"
Aug 03 16:34:58.988358 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.988342    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc380932-6576-4d70-98df-b1625d8abf6a" path="/var/lib/kubelet/pods/cc380932-6576-4d70-98df-b1625d8abf6a/volumes"
Aug 03 16:34:58.988839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:58.988816    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9eb0025-3014-4a2e-9b03-3333ae461d2d" path="/var/lib/kubelet/pods/f9eb0025-3014-4a2e-9b03-3333ae461d2d/volumes"
Aug 03 16:34:59.239686 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:34:59.239598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:01.255466 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:01.255421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:01.348121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:01.348085    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:02.334557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.334513    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"]
Aug 03 16:35:02.335078 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335063    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9e1b405-6e16-471f-8b8b-12f5847bcba2" containerName="payload-pre-processing"
Aug 03 16:35:02.335123 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335081    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9e1b405-6e16-471f-8b8b-12f5847bcba2" containerName="payload-pre-processing"
Aug 03 16:35:02.335123 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335116    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9eb0025-3014-4a2e-9b03-3333ae461d2d" containerName="payload-processing"
Aug 03 16:35:02.335188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335125    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9eb0025-3014-4a2e-9b03-3333ae461d2d" containerName="payload-processing"
Aug 03 16:35:02.335188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335137    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc380932-6576-4d70-98df-b1625d8abf6a" containerName="maas-api"
Aug 03 16:35:02.335188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335146    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc380932-6576-4d70-98df-b1625d8abf6a" containerName="maas-api"
Aug 03 16:35:02.335188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335154    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37efec57-39b8-45e4-a5be-e1892e753dd1" containerName="revoke-keys"
Aug 03 16:35:02.335188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335162    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37efec57-39b8-45e4-a5be-e1892e753dd1" containerName="revoke-keys"
Aug 03 16:35:02.335345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335243    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37efec57-39b8-45e4-a5be-e1892e753dd1" containerName="revoke-keys"
Aug 03 16:35:02.335345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335253    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9e1b405-6e16-471f-8b8b-12f5847bcba2" containerName="payload-pre-processing"
Aug 03 16:35:02.335345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335263    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9eb0025-3014-4a2e-9b03-3333ae461d2d" containerName="payload-processing"
Aug 03 16:35:02.335345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.335272    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc380932-6576-4d70-98df-b1625d8abf6a" containerName="maas-api"
Aug 03 16:35:02.340453 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.340431    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.343007 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.342987    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-fed81e80-serving-cert\""
Aug 03 16:35:02.349274 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.349249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"]
Aug 03 16:35:02.372192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.372168    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:02.442178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.442142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c7591a-4e01-449f-b9d5-89717da7bfe5-openshift-service-ca\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.442360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.442227    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.442360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.442258    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdqmd\" (UniqueName: \"kubernetes.io/projected/35c7591a-4e01-449f-b9d5-89717da7bfe5-kube-api-access-vdqmd\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.542947 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.542915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c7591a-4e01-449f-b9d5-89717da7bfe5-openshift-service-ca\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.543110 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.542976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.543110 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.543006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdqmd\" (UniqueName: \"kubernetes.io/projected/35c7591a-4e01-449f-b9d5-89717da7bfe5-kube-api-access-vdqmd\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.543183 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:35:02.543123    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-serving-cert: secret "maas-api-e2e-derive-fed81e80-serving-cert" not found
Aug 03 16:35:02.543218 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:35:02.543188    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls podName:35c7591a-4e01-449f-b9d5-89717da7bfe5 nodeName:}" failed. No retries permitted until 2026-08-03 16:35:03.043168316 +0000 UTC m=+1862.592454623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls") pod "maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" (UID: "35c7591a-4e01-449f-b9d5-89717da7bfe5") : secret "maas-api-e2e-derive-fed81e80-serving-cert" not found
Aug 03 16:35:02.543557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.543523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c7591a-4e01-449f-b9d5-89717da7bfe5-openshift-service-ca\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.554634 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.554608    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdqmd\" (UniqueName: \"kubernetes.io/projected/35c7591a-4e01-449f-b9d5-89717da7bfe5-kube-api-access-vdqmd\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:02.651903 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.651821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:02.945427 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:02.945340    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:03.042295 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.042254    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:03.047260 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.047231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:03.049285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.049260    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"]
Aug 03 16:35:03.049922 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.049896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls\") pod \"maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:03.053237 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.053220    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.059555 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.059514    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-fed81e80\""
Aug 03 16:35:03.059709 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.059692    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-fed81e80-dockercfg-fzwnv\""
Aug 03 16:35:03.068244 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.068220    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"]
Aug 03 16:35:03.125766 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.125735    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"]
Aug 03 16:35:03.129206 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.129190    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.144518 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.144489    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"]
Aug 03 16:35:03.148301 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.148275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwlf2\" (UniqueName: \"kubernetes.io/projected/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-kube-api-access-pwlf2\") pod \"payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") " pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.148441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.148357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-plugins-config-volume\") pod \"payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") " pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.249250 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.249165    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwlf2\" (UniqueName: \"kubernetes.io/projected/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-kube-api-access-pwlf2\") pod \"payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") " pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.249250 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.249217    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.249250 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.249245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txhbt\" (UniqueName: \"kubernetes.io/projected/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-kube-api-access-txhbt\") pod \"payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.249479 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.249267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-plugins-config-volume\") pod \"payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") " pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.250023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.250003    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-plugins-config-volume\") pod \"payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") " pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.253152 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.253127    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:03.258355 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.258332    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwlf2\" (UniqueName: \"kubernetes.io/projected/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-kube-api-access-pwlf2\") pod \"payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") " pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.350497 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.350464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txhbt\" (UniqueName: \"kubernetes.io/projected/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-kube-api-access-txhbt\") pod \"payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.350937 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.350603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.351513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.351483    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.361476 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.361449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txhbt\" (UniqueName: \"kubernetes.io/projected/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-kube-api-access-txhbt\") pod \"payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.368890 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.368145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.402192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.402163    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"]
Aug 03 16:35:03.403725 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:35:03.403694    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c7591a_4e01_449f_b9d5_89717da7bfe5.slice/crio-4cfb56e050cb4b5ed729e9ebd398a744a69b24aa24b14519d049cfd7633a9091 WatchSource:0}: Error finding container 4cfb56e050cb4b5ed729e9ebd398a744a69b24aa24b14519d049cfd7633a9091: Status 404 returned error can't find the container with id 4cfb56e050cb4b5ed729e9ebd398a744a69b24aa24b14519d049cfd7633a9091
Aug 03 16:35:03.439197 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.439170    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.531414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.531382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"]
Aug 03 16:35:03.533380 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:35:03.533357    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7db6051_9ed5_4a6e_998b_fdb0adc49e4f.slice/crio-90c3f156c708153502edba1497b9caa03d0af5c69528ade60c404ffd500acdb5 WatchSource:0}: Error finding container 90c3f156c708153502edba1497b9caa03d0af5c69528ade60c404ffd500acdb5: Status 404 returned error can't find the container with id 90c3f156c708153502edba1497b9caa03d0af5c69528ade60c404ffd500acdb5
Aug 03 16:35:03.598427 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.598401    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"]
Aug 03 16:35:03.601220 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:35:03.601178    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9d2aee_3ce7_4064_a0c4_d3eec6dd038f.slice/crio-a07957e165da51bda7d17eaa8346268b6a6cd48f4511b78ae156df609b353592 WatchSource:0}: Error finding container a07957e165da51bda7d17eaa8346268b6a6cd48f4511b78ae156df609b353592: Status 404 returned error can't find the container with id a07957e165da51bda7d17eaa8346268b6a6cd48f4511b78ae156df609b353592
Aug 03 16:35:03.980374 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.980341    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" event={"ID":"35c7591a-4e01-449f-b9d5-89717da7bfe5","Type":"ContainerStarted","Data":"4cfb56e050cb4b5ed729e9ebd398a744a69b24aa24b14519d049cfd7633a9091"}
Aug 03 16:35:03.981876 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.981853    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs" event={"ID":"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f","Type":"ContainerStarted","Data":"413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a"}
Aug 03 16:35:03.982138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.981892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs" event={"ID":"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f","Type":"ContainerStarted","Data":"90c3f156c708153502edba1497b9caa03d0af5c69528ade60c404ffd500acdb5"}
Aug 03 16:35:03.982138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.981920    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:03.983361 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.983338    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw" event={"ID":"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f","Type":"ContainerStarted","Data":"a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9"}
Aug 03 16:35:03.983451 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.983380    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw" event={"ID":"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f","Type":"ContainerStarted","Data":"a07957e165da51bda7d17eaa8346268b6a6cd48f4511b78ae156df609b353592"}
Aug 03 16:35:03.983451 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.983421    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:03.998886 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:03.998834    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs" podStartSLOduration=0.998815207 podStartE2EDuration="998.815207ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:35:03.997900174 +0000 UTC m=+1863.547186504" watchObservedRunningTime="2026-08-03 16:35:03.998815207 +0000 UTC m=+1863.548101524"
Aug 03 16:35:04.013747 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:04.013703    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw" podStartSLOduration=1.013689852 podStartE2EDuration="1.013689852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:35:04.013342151 +0000 UTC m=+1863.562628468" watchObservedRunningTime="2026-08-03 16:35:04.013689852 +0000 UTC m=+1863.562976170"
Aug 03 16:35:04.990002 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:04.989963    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" event={"ID":"35c7591a-4e01-449f-b9d5-89717da7bfe5","Type":"ContainerStarted","Data":"c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062"}
Aug 03 16:35:04.990553 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:04.990315    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:05.008901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:05.008849    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" podStartSLOduration=2.09478803 podStartE2EDuration="3.008834896s" totalImagesPullingTime="914.046866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:35:05.007333762 +0000 UTC m=+1864.556620080" watchObservedRunningTime="2026-08-03 16:35:05.008834896 +0000 UTC m=+1864.558121213"
Aug 03 16:35:11.001728 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:11.001697    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:14.991832 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:14.991804    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:14.992218 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:14.991946    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:26.150325 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.150243    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"]
Aug 03 16:35:26.154891 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.154868    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:26.157469 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.157446    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:35:26.161549 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.161502    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"]
Aug 03 16:35:26.249912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.249877    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5482398b-1135-473c-96ed-e9904d12b6d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb\" (UID: \"5482398b-1135-473c-96ed-e9904d12b6d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:26.350773 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.350739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5482398b-1135-473c-96ed-e9904d12b6d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb\" (UID: \"5482398b-1135-473c-96ed-e9904d12b6d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:26.351385 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.351359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5482398b-1135-473c-96ed-e9904d12b6d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb\" (UID: \"5482398b-1135-473c-96ed-e9904d12b6d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:26.468057 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.467970    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:26.602071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:26.602045    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"]
Aug 03 16:35:26.603879 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:35:26.603854    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5482398b_1135_473c_96ed_e9904d12b6d3.slice/crio-12382bbe16f4d77d3c82f785ddda3d531a7b69a17d8a7353657773afe30d2cd1 WatchSource:0}: Error finding container 12382bbe16f4d77d3c82f785ddda3d531a7b69a17d8a7353657773afe30d2cd1: Status 404 returned error can't find the container with id 12382bbe16f4d77d3c82f785ddda3d531a7b69a17d8a7353657773afe30d2cd1
Aug 03 16:35:27.103101 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:27.103027    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb" event={"ID":"5482398b-1135-473c-96ed-e9904d12b6d3","Type":"ContainerStarted","Data":"093068416fd0f9ec459388f2624e5302adacd6209985ec6d5bbe7f3127798a59"}
Aug 03 16:35:27.103101 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:27.103086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb" event={"ID":"5482398b-1135-473c-96ed-e9904d12b6d3","Type":"ContainerStarted","Data":"12382bbe16f4d77d3c82f785ddda3d531a7b69a17d8a7353657773afe30d2cd1"}
Aug 03 16:35:27.119933 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:27.119885    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb" podStartSLOduration=1.119869452 podStartE2EDuration="1.119869452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:35:27.118354242 +0000 UTC m=+1886.667640558" watchObservedRunningTime="2026-08-03 16:35:27.119869452 +0000 UTC m=+1886.669155825"
Aug 03 16:35:28.109657 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:28.109625    2560 generic.go:356] "Generic (PLEG): container finished" podID="5482398b-1135-473c-96ed-e9904d12b6d3" containerID="093068416fd0f9ec459388f2624e5302adacd6209985ec6d5bbe7f3127798a59" exitCode=0
Aug 03 16:35:28.110043 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:28.109714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb" event={"ID":"5482398b-1135-473c-96ed-e9904d12b6d3","Type":"ContainerDied","Data":"093068416fd0f9ec459388f2624e5302adacd6209985ec6d5bbe7f3127798a59"}
Aug 03 16:35:29.262387 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:29.262363    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:29.376351 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:29.376282    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5482398b-1135-473c-96ed-e9904d12b6d3-maas-api-service-ca\") pod \"5482398b-1135-473c-96ed-e9904d12b6d3\" (UID: \"5482398b-1135-473c-96ed-e9904d12b6d3\") "
Aug 03 16:35:29.376637 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:29.376616    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5482398b-1135-473c-96ed-e9904d12b6d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5482398b-1135-473c-96ed-e9904d12b6d3" (UID: "5482398b-1135-473c-96ed-e9904d12b6d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:35:29.477546 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:29.477496    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5482398b-1135-473c-96ed-e9904d12b6d3-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:30.120872 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:30.120843    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"
Aug 03 16:35:30.121035 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:30.120881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb" event={"ID":"5482398b-1135-473c-96ed-e9904d12b6d3","Type":"ContainerDied","Data":"12382bbe16f4d77d3c82f785ddda3d531a7b69a17d8a7353657773afe30d2cd1"}
Aug 03 16:35:30.121035 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:30.120928    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12382bbe16f4d77d3c82f785ddda3d531a7b69a17d8a7353657773afe30d2cd1"
Aug 03 16:35:33.096517 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.096477    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"]
Aug 03 16:35:33.096942 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.096764    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" podUID="35c7591a-4e01-449f-b9d5-89717da7bfe5" containerName="maas-api" containerID="cri-o://c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062" gracePeriod=30
Aug 03 16:35:33.146663 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.146632    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"]
Aug 03 16:35:33.146925 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.146869    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs" podUID="b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" containerName="payload-processing" containerID="cri-o://413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a" gracePeriod=30
Aug 03 16:35:33.152867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.152834    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"]
Aug 03 16:35:33.153257 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.153199    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw" podUID="0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" containerName="payload-pre-processing" containerID="cri-o://a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9" gracePeriod=30
Aug 03 16:35:33.157726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.157699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:33.254743 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.254718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:33.303358 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.303303    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"]
Aug 03 16:35:33.308543 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.308500    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fed81e80-2hrgb"]
Aug 03 16:35:33.329497 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.329401    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:33.423130 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.423101    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:33.460623 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.460599    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:33.469734 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.469708    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:33.481513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.481485    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:33.514860 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.514830    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txhbt\" (UniqueName: \"kubernetes.io/projected/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-kube-api-access-txhbt\") pod \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") "
Aug 03 16:35:33.515014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.514881    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls\") pod \"35c7591a-4e01-449f-b9d5-89717da7bfe5\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") "
Aug 03 16:35:33.515014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.514903    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-plugins-config-volume\") pod \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\" (UID: \"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f\") "
Aug 03 16:35:33.515014 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.514999    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdqmd\" (UniqueName: \"kubernetes.io/projected/35c7591a-4e01-449f-b9d5-89717da7bfe5-kube-api-access-vdqmd\") pod \"35c7591a-4e01-449f-b9d5-89717da7bfe5\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") "
Aug 03 16:35:33.515196 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515081    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c7591a-4e01-449f-b9d5-89717da7bfe5-openshift-service-ca\") pod \"35c7591a-4e01-449f-b9d5-89717da7bfe5\" (UID: \"35c7591a-4e01-449f-b9d5-89717da7bfe5\") "
Aug 03 16:35:33.515196 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515114    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwlf2\" (UniqueName: \"kubernetes.io/projected/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-kube-api-access-pwlf2\") pod \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") "
Aug 03 16:35:33.515196 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515138    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-plugins-config-volume\") pod \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\" (UID: \"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f\") "
Aug 03 16:35:33.515359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515254    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" (UID: "0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:35:33.515557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515514    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c7591a-4e01-449f-b9d5-89717da7bfe5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35c7591a-4e01-449f-b9d5-89717da7bfe5" (UID: "35c7591a-4e01-449f-b9d5-89717da7bfe5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:35:33.515660 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515577    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" (UID: "b7db6051-9ed5-4a6e-998b-fdb0adc49e4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:35:33.515722 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515670    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.515722 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515688    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c7591a-4e01-449f-b9d5-89717da7bfe5-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.515722 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.515702    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.517283 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.517262    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c7591a-4e01-449f-b9d5-89717da7bfe5-kube-api-access-vdqmd" (OuterVolumeSpecName: "kube-api-access-vdqmd") pod "35c7591a-4e01-449f-b9d5-89717da7bfe5" (UID: "35c7591a-4e01-449f-b9d5-89717da7bfe5"). InnerVolumeSpecName "kube-api-access-vdqmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:35:33.517374 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.517291    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-kube-api-access-txhbt" (OuterVolumeSpecName: "kube-api-access-txhbt") pod "0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" (UID: "0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f"). InnerVolumeSpecName "kube-api-access-txhbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:35:33.517374 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.517296    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35c7591a-4e01-449f-b9d5-89717da7bfe5" (UID: "35c7591a-4e01-449f-b9d5-89717da7bfe5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:35:33.517799 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.517782    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-kube-api-access-pwlf2" (OuterVolumeSpecName: "kube-api-access-pwlf2") pod "b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" (UID: "b7db6051-9ed5-4a6e-998b-fdb0adc49e4f"). InnerVolumeSpecName "kube-api-access-pwlf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:35:33.616473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.616383    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdqmd\" (UniqueName: \"kubernetes.io/projected/35c7591a-4e01-449f-b9d5-89717da7bfe5-kube-api-access-vdqmd\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.616473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.616416    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwlf2\" (UniqueName: \"kubernetes.io/projected/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f-kube-api-access-pwlf2\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.616473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.616429    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txhbt\" (UniqueName: \"kubernetes.io/projected/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f-kube-api-access-txhbt\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.616473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.616442    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c7591a-4e01-449f-b9d5-89717da7bfe5-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:35:33.693201 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:33.693169    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:34.142289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.142262    2560 generic.go:356] "Generic (PLEG): container finished" podID="0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" containerID="a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9" exitCode=0
Aug 03 16:35:34.142707 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.142307    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"
Aug 03 16:35:34.142707 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.142348    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw" event={"ID":"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f","Type":"ContainerDied","Data":"a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9"}
Aug 03 16:35:34.142707 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.142405    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw" event={"ID":"0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f","Type":"ContainerDied","Data":"a07957e165da51bda7d17eaa8346268b6a6cd48f4511b78ae156df609b353592"}
Aug 03 16:35:34.142707 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.142437    2560 scope.go:117] "RemoveContainer" containerID="a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9"
Aug 03 16:35:34.143791 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.143773    2560 generic.go:356] "Generic (PLEG): container finished" podID="35c7591a-4e01-449f-b9d5-89717da7bfe5" containerID="c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062" exitCode=0
Aug 03 16:35:34.143906 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.143803    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" event={"ID":"35c7591a-4e01-449f-b9d5-89717da7bfe5","Type":"ContainerDied","Data":"c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062"}
Aug 03 16:35:34.143906 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.143823    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"
Aug 03 16:35:34.143906 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.143862    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7" event={"ID":"35c7591a-4e01-449f-b9d5-89717da7bfe5","Type":"ContainerDied","Data":"4cfb56e050cb4b5ed729e9ebd398a744a69b24aa24b14519d049cfd7633a9091"}
Aug 03 16:35:34.145219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.145189    2560 generic.go:356] "Generic (PLEG): container finished" podID="b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" containerID="413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a" exitCode=0
Aug 03 16:35:34.145333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.145220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs" event={"ID":"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f","Type":"ContainerDied","Data":"413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a"}
Aug 03 16:35:34.145333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.145235    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"
Aug 03 16:35:34.145333 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.145249    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs" event={"ID":"b7db6051-9ed5-4a6e-998b-fdb0adc49e4f","Type":"ContainerDied","Data":"90c3f156c708153502edba1497b9caa03d0af5c69528ade60c404ffd500acdb5"}
Aug 03 16:35:34.153164 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.153146    2560 scope.go:117] "RemoveContainer" containerID="a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9"
Aug 03 16:35:34.153448 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:35:34.153425    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9\": container with ID starting with a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9 not found: ID does not exist" containerID="a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9"
Aug 03 16:35:34.153580 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.153454    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9"} err="failed to get container status \"a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9\": rpc error: code = NotFound desc = could not find container \"a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9\": container with ID starting with a9fa44f85e27de17533563799dd5aee17567636ea9ad8d0efa5f61e73f4a7de9 not found: ID does not exist"
Aug 03 16:35:34.153580 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.153471    2560 scope.go:117] "RemoveContainer" containerID="c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062"
Aug 03 16:35:34.157968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.157951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:34.162390 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.162371    2560 scope.go:117] "RemoveContainer" containerID="c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062"
Aug 03 16:35:34.162708 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:35:34.162687    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062\": container with ID starting with c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062 not found: ID does not exist" containerID="c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062"
Aug 03 16:35:34.162761 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.162715    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062"} err="failed to get container status \"c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062\": rpc error: code = NotFound desc = could not find container \"c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062\": container with ID starting with c22c793d8b2f56714b684c98e4d9cb83533261d9bd5ceb2b8076bb502bcae062 not found: ID does not exist"
Aug 03 16:35:34.162761 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.162731    2560 scope.go:117] "RemoveContainer" containerID="413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a"
Aug 03 16:35:34.170247 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.170229    2560 scope.go:117] "RemoveContainer" containerID="413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a"
Aug 03 16:35:34.170463 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:35:34.170445    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a\": container with ID starting with 413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a not found: ID does not exist" containerID="413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a"
Aug 03 16:35:34.170522 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.170468    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a"} err="failed to get container status \"413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a\": rpc error: code = NotFound desc = could not find container \"413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a\": container with ID starting with 413fdf0cae885d6611396785d255a9ceffde4a3ffca37a4a511699a9ebd8367a not found: ID does not exist"
Aug 03 16:35:34.174124 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.174104    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"]
Aug 03 16:35:34.181561 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.181514    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-fed81e80-5d7bf89455-z2xc7"]
Aug 03 16:35:34.195871 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.195847    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"]
Aug 03 16:35:34.199537 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.199506    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fed81e80-7776bd5b44-5s5qw"]
Aug 03 16:35:34.213351 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.213328    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"]
Aug 03 16:35:34.215134 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.215115    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fed81e80-69df869b9d-xt7hs"]
Aug 03 16:35:34.256019 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.255994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:34.356853 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.356813    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:34.986577 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.986544    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" path="/var/lib/kubelet/pods/0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f/volumes"
Aug 03 16:35:34.986954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.986940    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c7591a-4e01-449f-b9d5-89717da7bfe5" path="/var/lib/kubelet/pods/35c7591a-4e01-449f-b9d5-89717da7bfe5/volumes"
Aug 03 16:35:34.987230 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.987218    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5482398b-1135-473c-96ed-e9904d12b6d3" path="/var/lib/kubelet/pods/5482398b-1135-473c-96ed-e9904d12b6d3/volumes"
Aug 03 16:35:34.987540 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:34.987513    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" path="/var/lib/kubelet/pods/b7db6051-9ed5-4a6e-998b-fdb0adc49e4f/volumes"
Aug 03 16:35:46.066504 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:46.066472    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:52.745055 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:52.745006    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:52.846286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:52.846250    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:52.944042 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:52.944009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:53.044955 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:53.044928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:35:53.487095 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:35:53.487001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:07.234247 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:07.234214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:07.294848 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:07.294818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:19.467474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:19.467442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:25.783148 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:25.783089    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:25.847726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:25.847690    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:25.947353 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:25.947309    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:26.037827 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:26.037731    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:41.901162 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:41.901126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:47.740695 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:47.740663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:47.803128 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:47.803098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:47.885056 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:47.885020    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:47.985250 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:47.985219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:57.360347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:57.360264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:36:58.460280 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:36:58.460232    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:09.528339 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:09.528274    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:15.721781 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:15.721744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:15.857898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:15.857866    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:15.975803 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:15.975711    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:16.067993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:16.067956    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:16.719334 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:16.719299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:16.813416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:16.813385    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:16.916717 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:16.916680    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:18.381149 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:18.381113    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:24.994217 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:24.994173    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:25.104196 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:25.104153    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:25.781121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:25.781084    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:31.929836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:31.929799    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:32.091855 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:32.091820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:32.218586 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:32.218444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:32.487850 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:32.487741    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:33.339415 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:33.339379    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:35.708176 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:35.708137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:36.856680 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.856644    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"]
Aug 03 16:37:36.857225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857200    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c7591a-4e01-449f-b9d5-89717da7bfe5" containerName="maas-api"
Aug 03 16:37:36.857225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857227    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c7591a-4e01-449f-b9d5-89717da7bfe5" containerName="maas-api"
Aug 03 16:37:36.857362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857242    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" containerName="payload-processing"
Aug 03 16:37:36.857362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857252    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" containerName="payload-processing"
Aug 03 16:37:36.857362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857278    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5482398b-1135-473c-96ed-e9904d12b6d3" containerName="revoke-keys"
Aug 03 16:37:36.857362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857285    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5482398b-1135-473c-96ed-e9904d12b6d3" containerName="revoke-keys"
Aug 03 16:37:36.857362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857296    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" containerName="payload-pre-processing"
Aug 03 16:37:36.857362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857301    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" containerName="payload-pre-processing"
Aug 03 16:37:36.857698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857377    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9d2aee-3ce7-4064-a0c4-d3eec6dd038f" containerName="payload-pre-processing"
Aug 03 16:37:36.857698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857390    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c7591a-4e01-449f-b9d5-89717da7bfe5" containerName="maas-api"
Aug 03 16:37:36.857698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857402    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7db6051-9ed5-4a6e-998b-fdb0adc49e4f" containerName="payload-processing"
Aug 03 16:37:36.857698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.857411    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5482398b-1135-473c-96ed-e9904d12b6d3" containerName="revoke-keys"
Aug 03 16:37:36.860578 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.860556    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:36.863370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.863343    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-de4e0ed8-serving-cert\""
Aug 03 16:37:36.870818 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:36.870796    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"]
Aug 03 16:37:37.019265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.019232    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2bbb04-e421-460e-ada1-e2a7970953a3-openshift-service-ca\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.019461 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.019290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb2bbb04-e421-460e-ada1-e2a7970953a3-maas-api-tls\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.019461 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.019343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjnb6\" (UniqueName: \"kubernetes.io/projected/bb2bbb04-e421-460e-ada1-e2a7970953a3-kube-api-access-qjnb6\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.120202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.120110    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb2bbb04-e421-460e-ada1-e2a7970953a3-maas-api-tls\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.120202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.120146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjnb6\" (UniqueName: \"kubernetes.io/projected/bb2bbb04-e421-460e-ada1-e2a7970953a3-kube-api-access-qjnb6\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.120456 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.120332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2bbb04-e421-460e-ada1-e2a7970953a3-openshift-service-ca\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.120959 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.120939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2bbb04-e421-460e-ada1-e2a7970953a3-openshift-service-ca\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.122607 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.122581    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb2bbb04-e421-460e-ada1-e2a7970953a3-maas-api-tls\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.129196 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.129161    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjnb6\" (UniqueName: \"kubernetes.io/projected/bb2bbb04-e421-460e-ada1-e2a7970953a3-kube-api-access-qjnb6\") pod \"maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.173665 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.173621    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:37.491574 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.491470    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"]
Aug 03 16:37:37.495408 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.495380    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.498317 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.498286    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-de4e0ed8\""
Aug 03 16:37:37.498451 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.498405    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-de4e0ed8-dockercfg-zfxgn\""
Aug 03 16:37:37.506502 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.506478    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"]
Aug 03 16:37:37.509381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.509326    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:37:37.523668 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.523636    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"]
Aug 03 16:37:37.527358 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.527322    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"]
Aug 03 16:37:37.527480 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.527343    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.530186 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:37:37.530154    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2bbb04_e421_460e_ada1_e2a7970953a3.slice/crio-c53d525ce5289ecc6f9a554bc7fc30a9c9abd86ec30225fcee8d9d38426a12b7 WatchSource:0}: Error finding container c53d525ce5289ecc6f9a554bc7fc30a9c9abd86ec30225fcee8d9d38426a12b7: Status 404 returned error can't find the container with id c53d525ce5289ecc6f9a554bc7fc30a9c9abd86ec30225fcee8d9d38426a12b7
Aug 03 16:37:37.539463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.539403    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"]
Aug 03 16:37:37.624861 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.624820    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535ed176-34e0-47de-a6d3-12fb9a88a79f-plugins-config-volume\") pod \"payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") " pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.625084 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.624908    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86fb4ee-b39f-4c63-91ea-50f471a39402-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.625084 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.624947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkw5b\" (UniqueName: \"kubernetes.io/projected/e86fb4ee-b39f-4c63-91ea-50f471a39402-kube-api-access-dkw5b\") pod \"payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.625084 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.624972    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpjj\" (UniqueName: \"kubernetes.io/projected/535ed176-34e0-47de-a6d3-12fb9a88a79f-kube-api-access-6mpjj\") pod \"payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") " pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.725795 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.725758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86fb4ee-b39f-4c63-91ea-50f471a39402-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.725988 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.725815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkw5b\" (UniqueName: \"kubernetes.io/projected/e86fb4ee-b39f-4c63-91ea-50f471a39402-kube-api-access-dkw5b\") pod \"payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.725988 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.725850    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpjj\" (UniqueName: \"kubernetes.io/projected/535ed176-34e0-47de-a6d3-12fb9a88a79f-kube-api-access-6mpjj\") pod \"payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") " pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.725988 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.725880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535ed176-34e0-47de-a6d3-12fb9a88a79f-plugins-config-volume\") pod \"payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") " pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.726608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.726584    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535ed176-34e0-47de-a6d3-12fb9a88a79f-plugins-config-volume\") pod \"payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") " pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.726608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.726596    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86fb4ee-b39f-4c63-91ea-50f471a39402-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.734936 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.734911    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkw5b\" (UniqueName: \"kubernetes.io/projected/e86fb4ee-b39f-4c63-91ea-50f471a39402-kube-api-access-dkw5b\") pod \"payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.735056 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.734960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpjj\" (UniqueName: \"kubernetes.io/projected/535ed176-34e0-47de-a6d3-12fb9a88a79f-kube-api-access-6mpjj\") pod \"payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") " pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.774356 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.774270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt" event={"ID":"bb2bbb04-e421-460e-ada1-e2a7970953a3","Type":"ContainerStarted","Data":"c53d525ce5289ecc6f9a554bc7fc30a9c9abd86ec30225fcee8d9d38426a12b7"}
Aug 03 16:37:37.807645 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.807606    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:37.849471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.849079    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:37.961273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:37.961241    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"]
Aug 03 16:37:37.963127 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:37:37.963093    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86fb4ee_b39f_4c63_91ea_50f471a39402.slice/crio-02e933ee5b6fba36ba131896ab7993f00c8abbe8a6b791953e1949474fc94de7 WatchSource:0}: Error finding container 02e933ee5b6fba36ba131896ab7993f00c8abbe8a6b791953e1949474fc94de7: Status 404 returned error can't find the container with id 02e933ee5b6fba36ba131896ab7993f00c8abbe8a6b791953e1949474fc94de7
Aug 03 16:37:38.010059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.008186    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"]
Aug 03 16:37:38.015053 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:37:38.012710    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535ed176_34e0_47de_a6d3_12fb9a88a79f.slice/crio-586f5ff4fd18237c3702f204a61a5511d2da694e7915994e5aafedbd94009cf7 WatchSource:0}: Error finding container 586f5ff4fd18237c3702f204a61a5511d2da694e7915994e5aafedbd94009cf7: Status 404 returned error can't find the container with id 586f5ff4fd18237c3702f204a61a5511d2da694e7915994e5aafedbd94009cf7
Aug 03 16:37:38.780959 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.780926    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt" event={"ID":"bb2bbb04-e421-460e-ada1-e2a7970953a3","Type":"ContainerStarted","Data":"f2baee2023ae444e2e940f227d7f2e1cc29936a250973b1147f8c3073d7725d2"}
Aug 03 16:37:38.781163 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.781044    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:38.782600 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.782574    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g" event={"ID":"e86fb4ee-b39f-4c63-91ea-50f471a39402","Type":"ContainerStarted","Data":"e9e4ac5bee87b3bc4613ec235033d57a4c3dc2eb944572dc88294dc7335c784e"}
Aug 03 16:37:38.782740 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.782615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g" event={"ID":"e86fb4ee-b39f-4c63-91ea-50f471a39402","Type":"ContainerStarted","Data":"02e933ee5b6fba36ba131896ab7993f00c8abbe8a6b791953e1949474fc94de7"}
Aug 03 16:37:38.782740 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.782645    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:38.784054 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.784032    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz" event={"ID":"535ed176-34e0-47de-a6d3-12fb9a88a79f","Type":"ContainerStarted","Data":"d706ba887ccec435d18ca1d7839500d22300eebf33fb2fdf31a2ba99d1283858"}
Aug 03 16:37:38.784177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.784078    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz" event={"ID":"535ed176-34e0-47de-a6d3-12fb9a88a79f","Type":"ContainerStarted","Data":"586f5ff4fd18237c3702f204a61a5511d2da694e7915994e5aafedbd94009cf7"}
Aug 03 16:37:38.784177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.784137    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:37:38.800664 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.800602    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt" podStartSLOduration=2.292062454 podStartE2EDuration="2.80058039s" totalImagesPullingTime="508.517936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:37:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:37:38.798411681 +0000 UTC m=+2018.347698022" watchObservedRunningTime="2026-08-03 16:37:38.80058039 +0000 UTC m=+2018.349866712"
Aug 03 16:37:38.819662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.819610    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g" podStartSLOduration=1.8195915679999999 podStartE2EDuration="1.819591568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:37:38.817348626 +0000 UTC m=+2018.366634943" watchObservedRunningTime="2026-08-03 16:37:38.819591568 +0000 UTC m=+2018.368877883"
Aug 03 16:37:38.861039 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:38.860980    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz" podStartSLOduration=1.8609640490000001 podStartE2EDuration="1.860964049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:37:38.858475108 +0000 UTC m=+2018.407761425" watchObservedRunningTime="2026-08-03 16:37:38.860964049 +0000 UTC m=+2018.410250377"
Aug 03 16:37:44.796741 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:44.796712    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:37:49.792201 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:49.792171    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:37:49.792752 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:37:49.792256    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:38:00.695315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:00.695257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:01.377585 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:01.377552    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:01.472181 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:01.472118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:01.854150 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:01.854118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:07.825197 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:07.825154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:07.925225 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:07.925190    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:08.222594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.222502    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"]
Aug 03 16:38:08.226229 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.226210    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:08.229035 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.229010    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:38:08.233332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.233305    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"]
Aug 03 16:38:08.299963 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.299914    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh\" (UID: \"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:08.401273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.401231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh\" (UID: \"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:08.401915 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.401896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh\" (UID: \"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:08.539057 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.539017    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:08.679003 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.678978    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"]
Aug 03 16:38:08.682282 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:38:08.682253    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c8ddd7_0f1c_4fe3_a864_a6e5b4c8bbf7.slice/crio-766d0e8d813fb20f311b52ad9ee971788c99986552e8b03389d7eb4dffe0daef WatchSource:0}: Error finding container 766d0e8d813fb20f311b52ad9ee971788c99986552e8b03389d7eb4dffe0daef: Status 404 returned error can't find the container with id 766d0e8d813fb20f311b52ad9ee971788c99986552e8b03389d7eb4dffe0daef
Aug 03 16:38:08.953758 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:08.953719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh" event={"ID":"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7","Type":"ContainerStarted","Data":"766d0e8d813fb20f311b52ad9ee971788c99986552e8b03389d7eb4dffe0daef"}
Aug 03 16:38:09.960719 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:09.960685    2560 generic.go:356] "Generic (PLEG): container finished" podID="36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7" containerID="e0c3d063ef62905defcc76d1dc0d716f16ea9d807756104aaf5ddbe2f1fb237a" exitCode=0
Aug 03 16:38:09.961184 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:09.960731    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh" event={"ID":"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7","Type":"ContainerDied","Data":"e0c3d063ef62905defcc76d1dc0d716f16ea9d807756104aaf5ddbe2f1fb237a"}
Aug 03 16:38:11.105618 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.105578    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:11.228462 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.228371    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7-maas-api-service-ca\") pod \"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7\" (UID: \"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7\") "
Aug 03 16:38:11.228802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.228778    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7" (UID: "36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:38:11.329882 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.329848    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:11.973484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.973454    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"
Aug 03 16:38:11.973688 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.973486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh" event={"ID":"36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7","Type":"ContainerDied","Data":"766d0e8d813fb20f311b52ad9ee971788c99986552e8b03389d7eb4dffe0daef"}
Aug 03 16:38:11.973688 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:11.973546    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="766d0e8d813fb20f311b52ad9ee971788c99986552e8b03389d7eb4dffe0daef"
Aug 03 16:38:18.775037 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.775000    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"]
Aug 03 16:38:18.775509 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.775261    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt" podUID="bb2bbb04-e421-460e-ada1-e2a7970953a3" containerName="maas-api" containerID="cri-o://f2baee2023ae444e2e940f227d7f2e1cc29936a250973b1147f8c3073d7725d2" gracePeriod=30
Aug 03 16:38:18.799144 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.799114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:18.830048 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.829986    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"]
Aug 03 16:38:18.830594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.830565    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz" podUID="535ed176-34e0-47de-a6d3-12fb9a88a79f" containerName="payload-processing" containerID="cri-o://d706ba887ccec435d18ca1d7839500d22300eebf33fb2fdf31a2ba99d1283858" gracePeriod=30
Aug 03 16:38:18.840339 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.840307    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"]
Aug 03 16:38:18.840633 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.840605    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g" podUID="e86fb4ee-b39f-4c63-91ea-50f471a39402" containerName="payload-pre-processing" containerID="cri-o://e9e4ac5bee87b3bc4613ec235033d57a4c3dc2eb944572dc88294dc7335c784e" gracePeriod=30
Aug 03 16:38:18.857061 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.857034    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:18.957960 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.957928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:18.967644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.967516    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"]
Aug 03 16:38:18.971713 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.971682    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-de4e0ed8-k9mwh"]
Aug 03 16:38:18.990020 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:18.989848    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7" path="/var/lib/kubelet/pods/36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7/volumes"
Aug 03 16:38:19.012320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.012242    2560 generic.go:356] "Generic (PLEG): container finished" podID="535ed176-34e0-47de-a6d3-12fb9a88a79f" containerID="d706ba887ccec435d18ca1d7839500d22300eebf33fb2fdf31a2ba99d1283858" exitCode=0
Aug 03 16:38:19.012566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.012287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz" event={"ID":"535ed176-34e0-47de-a6d3-12fb9a88a79f","Type":"ContainerDied","Data":"d706ba887ccec435d18ca1d7839500d22300eebf33fb2fdf31a2ba99d1283858"}
Aug 03 16:38:19.015746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.015712    2560 generic.go:356] "Generic (PLEG): container finished" podID="bb2bbb04-e421-460e-ada1-e2a7970953a3" containerID="f2baee2023ae444e2e940f227d7f2e1cc29936a250973b1147f8c3073d7725d2" exitCode=0
Aug 03 16:38:19.015858 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.015800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt" event={"ID":"bb2bbb04-e421-460e-ada1-e2a7970953a3","Type":"ContainerDied","Data":"f2baee2023ae444e2e940f227d7f2e1cc29936a250973b1147f8c3073d7725d2"}
Aug 03 16:38:19.018274 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.017919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:19.018409 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.018307    2560 generic.go:356] "Generic (PLEG): container finished" podID="e86fb4ee-b39f-4c63-91ea-50f471a39402" containerID="e9e4ac5bee87b3bc4613ec235033d57a4c3dc2eb944572dc88294dc7335c784e" exitCode=0
Aug 03 16:38:19.018409 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.018341    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g" event={"ID":"e86fb4ee-b39f-4c63-91ea-50f471a39402","Type":"ContainerDied","Data":"e9e4ac5bee87b3bc4613ec235033d57a4c3dc2eb944572dc88294dc7335c784e"}
Aug 03 16:38:19.141772 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.141745    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:38:19.144721 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.144695    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:38:19.149032 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.149009    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:38:19.306395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306310    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2bbb04-e421-460e-ada1-e2a7970953a3-openshift-service-ca\") pod \"bb2bbb04-e421-460e-ada1-e2a7970953a3\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") "
Aug 03 16:38:19.306395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306353    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mpjj\" (UniqueName: \"kubernetes.io/projected/535ed176-34e0-47de-a6d3-12fb9a88a79f-kube-api-access-6mpjj\") pod \"535ed176-34e0-47de-a6d3-12fb9a88a79f\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") "
Aug 03 16:38:19.306395 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306380    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkw5b\" (UniqueName: \"kubernetes.io/projected/e86fb4ee-b39f-4c63-91ea-50f471a39402-kube-api-access-dkw5b\") pod \"e86fb4ee-b39f-4c63-91ea-50f471a39402\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") "
Aug 03 16:38:19.306668 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306446    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb2bbb04-e421-460e-ada1-e2a7970953a3-maas-api-tls\") pod \"bb2bbb04-e421-460e-ada1-e2a7970953a3\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") "
Aug 03 16:38:19.306668 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306656    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjnb6\" (UniqueName: \"kubernetes.io/projected/bb2bbb04-e421-460e-ada1-e2a7970953a3-kube-api-access-qjnb6\") pod \"bb2bbb04-e421-460e-ada1-e2a7970953a3\" (UID: \"bb2bbb04-e421-460e-ada1-e2a7970953a3\") "
Aug 03 16:38:19.306788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306730    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86fb4ee-b39f-4c63-91ea-50f471a39402-plugins-config-volume\") pod \"e86fb4ee-b39f-4c63-91ea-50f471a39402\" (UID: \"e86fb4ee-b39f-4c63-91ea-50f471a39402\") "
Aug 03 16:38:19.306788 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306765    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535ed176-34e0-47de-a6d3-12fb9a88a79f-plugins-config-volume\") pod \"535ed176-34e0-47de-a6d3-12fb9a88a79f\" (UID: \"535ed176-34e0-47de-a6d3-12fb9a88a79f\") "
Aug 03 16:38:19.307091 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.306969    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2bbb04-e421-460e-ada1-e2a7970953a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb2bbb04-e421-460e-ada1-e2a7970953a3" (UID: "bb2bbb04-e421-460e-ada1-e2a7970953a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:38:19.307227 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.307181    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e86fb4ee-b39f-4c63-91ea-50f471a39402-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e86fb4ee-b39f-4c63-91ea-50f471a39402" (UID: "e86fb4ee-b39f-4c63-91ea-50f471a39402"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:38:19.307227 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.307219    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2bbb04-e421-460e-ada1-e2a7970953a3-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:19.307352 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.307215    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/535ed176-34e0-47de-a6d3-12fb9a88a79f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "535ed176-34e0-47de-a6d3-12fb9a88a79f" (UID: "535ed176-34e0-47de-a6d3-12fb9a88a79f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:38:19.308737 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.308707    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2bbb04-e421-460e-ada1-e2a7970953a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb2bbb04-e421-460e-ada1-e2a7970953a3" (UID: "bb2bbb04-e421-460e-ada1-e2a7970953a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:38:19.308851 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.308777    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535ed176-34e0-47de-a6d3-12fb9a88a79f-kube-api-access-6mpjj" (OuterVolumeSpecName: "kube-api-access-6mpjj") pod "535ed176-34e0-47de-a6d3-12fb9a88a79f" (UID: "535ed176-34e0-47de-a6d3-12fb9a88a79f"). InnerVolumeSpecName "kube-api-access-6mpjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:38:19.308851 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.308809    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb2bbb04-e421-460e-ada1-e2a7970953a3-kube-api-access-qjnb6" (OuterVolumeSpecName: "kube-api-access-qjnb6") pod "bb2bbb04-e421-460e-ada1-e2a7970953a3" (UID: "bb2bbb04-e421-460e-ada1-e2a7970953a3"). InnerVolumeSpecName "kube-api-access-qjnb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:38:19.309004 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.308986    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86fb4ee-b39f-4c63-91ea-50f471a39402-kube-api-access-dkw5b" (OuterVolumeSpecName: "kube-api-access-dkw5b") pod "e86fb4ee-b39f-4c63-91ea-50f471a39402" (UID: "e86fb4ee-b39f-4c63-91ea-50f471a39402"). InnerVolumeSpecName "kube-api-access-dkw5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:38:19.395294 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.395251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:19.408539 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.408503    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e86fb4ee-b39f-4c63-91ea-50f471a39402-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:19.408698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.408555    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/535ed176-34e0-47de-a6d3-12fb9a88a79f-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:19.408698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.408565    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mpjj\" (UniqueName: \"kubernetes.io/projected/535ed176-34e0-47de-a6d3-12fb9a88a79f-kube-api-access-6mpjj\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:19.408698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.408574    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkw5b\" (UniqueName: \"kubernetes.io/projected/e86fb4ee-b39f-4c63-91ea-50f471a39402-kube-api-access-dkw5b\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:19.408698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.408583    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb2bbb04-e421-460e-ada1-e2a7970953a3-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:19.408698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:19.408591    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjnb6\" (UniqueName: \"kubernetes.io/projected/bb2bbb04-e421-460e-ada1-e2a7970953a3-kube-api-access-qjnb6\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:38:20.025286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.025256    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"
Aug 03 16:38:20.025793 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.025251    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt" event={"ID":"bb2bbb04-e421-460e-ada1-e2a7970953a3","Type":"ContainerDied","Data":"c53d525ce5289ecc6f9a554bc7fc30a9c9abd86ec30225fcee8d9d38426a12b7"}
Aug 03 16:38:20.025793 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.025407    2560 scope.go:117] "RemoveContainer" containerID="f2baee2023ae444e2e940f227d7f2e1cc29936a250973b1147f8c3073d7725d2"
Aug 03 16:38:20.028642 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.028200    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"
Aug 03 16:38:20.028642 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.028228    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g" event={"ID":"e86fb4ee-b39f-4c63-91ea-50f471a39402","Type":"ContainerDied","Data":"02e933ee5b6fba36ba131896ab7993f00c8abbe8a6b791953e1949474fc94de7"}
Aug 03 16:38:20.036376 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.036343    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz" event={"ID":"535ed176-34e0-47de-a6d3-12fb9a88a79f","Type":"ContainerDied","Data":"586f5ff4fd18237c3702f204a61a5511d2da694e7915994e5aafedbd94009cf7"}
Aug 03 16:38:20.036558 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.036375    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"
Aug 03 16:38:20.046111 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.045896    2560 scope.go:117] "RemoveContainer" containerID="e9e4ac5bee87b3bc4613ec235033d57a4c3dc2eb944572dc88294dc7335c784e"
Aug 03 16:38:20.055440 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.055402    2560 scope.go:117] "RemoveContainer" containerID="d706ba887ccec435d18ca1d7839500d22300eebf33fb2fdf31a2ba99d1283858"
Aug 03 16:38:20.057946 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.057922    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"]
Aug 03 16:38:20.061348 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.061327    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de4e0ed8-844c6c7d57-k26mt"]
Aug 03 16:38:20.072940 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.072915    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"]
Aug 03 16:38:20.077024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.077001    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-de4e0ed8-54dfb7dd64-9np7g"]
Aug 03 16:38:20.088273 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.088243    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"]
Aug 03 16:38:20.092616 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.092594    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-de4e0ed8-799c8486f8-jp5dz"]
Aug 03 16:38:20.988193 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.988144    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="535ed176-34e0-47de-a6d3-12fb9a88a79f" path="/var/lib/kubelet/pods/535ed176-34e0-47de-a6d3-12fb9a88a79f/volumes"
Aug 03 16:38:20.988701 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.988679    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb2bbb04-e421-460e-ada1-e2a7970953a3" path="/var/lib/kubelet/pods/bb2bbb04-e421-460e-ada1-e2a7970953a3/volumes"
Aug 03 16:38:20.989088 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:20.989069    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e86fb4ee-b39f-4c63-91ea-50f471a39402" path="/var/lib/kubelet/pods/e86fb4ee-b39f-4c63-91ea-50f471a39402/volumes"
Aug 03 16:38:25.552113 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:25.552039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:25.621832 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:25.621793    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:25.829901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:25.829802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:26.076977 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:26.076938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:26.218132 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:26.218035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:26.327081 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:26.327046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:26.536224 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:26.536198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:32.953867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:32.953825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:33.028609 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:33.028575    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:33.116106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:33.116072    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:33.339787 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:33.339730    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:39.534228 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:39.534190    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:38:39.594059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:38:39.594021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:39:01.064019 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:39:01.063987    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:39:01.072020 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:39:01.071998    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:39:22.397439 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:39:22.397404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:01.193250 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:01.193171    2560 scope.go:117] "RemoveContainer" containerID="dfa5786db48a2a7e180e4e8840758f265d8c9fdea9d73fbaf0ba160864ddee48"
Aug 03 16:40:10.894050 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:10.894012    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:42.296998 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:42.296961    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:42.607426 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:42.607341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:42.938347 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:42.938264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:45.790776 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:45.790744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:45.877178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:45.877145    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:46.871808 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.871776    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"]
Aug 03 16:40:46.872270 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872252    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86fb4ee-b39f-4c63-91ea-50f471a39402" containerName="payload-pre-processing"
Aug 03 16:40:46.872320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872272    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86fb4ee-b39f-4c63-91ea-50f471a39402" containerName="payload-pre-processing"
Aug 03 16:40:46.872320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872288    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2bbb04-e421-460e-ada1-e2a7970953a3" containerName="maas-api"
Aug 03 16:40:46.872320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872294    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2bbb04-e421-460e-ada1-e2a7970953a3" containerName="maas-api"
Aug 03 16:40:46.872320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872301    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="535ed176-34e0-47de-a6d3-12fb9a88a79f" containerName="payload-processing"
Aug 03 16:40:46.872320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872308    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="535ed176-34e0-47de-a6d3-12fb9a88a79f" containerName="payload-processing"
Aug 03 16:40:46.872320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872319    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7" containerName="revoke-keys"
Aug 03 16:40:46.872566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872324    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7" containerName="revoke-keys"
Aug 03 16:40:46.872566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872394    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86fb4ee-b39f-4c63-91ea-50f471a39402" containerName="payload-pre-processing"
Aug 03 16:40:46.872566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872402    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="535ed176-34e0-47de-a6d3-12fb9a88a79f" containerName="payload-processing"
Aug 03 16:40:46.872566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872409    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="36c8ddd7-0f1c-4fe3-a864-a6e5b4c8bbf7" containerName="revoke-keys"
Aug 03 16:40:46.872566 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.872417    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2bbb04-e421-460e-ada1-e2a7970953a3" containerName="maas-api"
Aug 03 16:40:46.875646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.875629    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:46.878153 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.878133    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-65805bcc-serving-cert\""
Aug 03 16:40:46.886448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.886420    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"]
Aug 03 16:40:46.909905 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:46.909871    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:47.008286 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.008253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.008465 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.008295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpbml\" (UniqueName: \"kubernetes.io/projected/42495b00-b56f-46da-8931-062be1e4a3b8-kube-api-access-kpbml\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.008465 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.008388    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42495b00-b56f-46da-8931-062be1e4a3b8-openshift-service-ca\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.109838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.109805    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.109838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.109849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpbml\" (UniqueName: \"kubernetes.io/projected/42495b00-b56f-46da-8931-062be1e4a3b8-kube-api-access-kpbml\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.110063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.109897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42495b00-b56f-46da-8931-062be1e4a3b8-openshift-service-ca\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.110063 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:40:47.109963    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-serving-cert: secret "maas-api-e2e-mt-65805bcc-serving-cert" not found
Aug 03 16:40:47.110063 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:40:47.110036    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls podName:42495b00-b56f-46da-8931-062be1e4a3b8 nodeName:}" failed. No retries permitted until 2026-08-03 16:40:47.610017664 +0000 UTC m=+2207.159303958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls") pod "maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" (UID: "42495b00-b56f-46da-8931-062be1e4a3b8") : secret "maas-api-e2e-mt-65805bcc-serving-cert" not found
Aug 03 16:40:47.110604 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.110582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42495b00-b56f-46da-8931-062be1e4a3b8-openshift-service-ca\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.122565 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.122500    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpbml\" (UniqueName: \"kubernetes.io/projected/42495b00-b56f-46da-8931-062be1e4a3b8-kube-api-access-kpbml\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.452458 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.452376    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:47.563282 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.563249    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:47.613667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.613625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.616008 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.615983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls\") pod \"maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.664292 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.664262    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:40:47.671204 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.671177    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"]
Aug 03 16:40:47.674801 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.674784    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.677630 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.677610    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-65805bcc\""
Aug 03 16:40:47.677735 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.677717    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-65805bcc-dockercfg-p2fs2\""
Aug 03 16:40:47.684873 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.684853    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"]
Aug 03 16:40:47.711021 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.710961    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"]
Aug 03 16:40:47.715069 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.715049    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.722965 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.722928    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"]
Aug 03 16:40:47.789283 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.789256    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:47.815044 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.815012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9913f895-35b7-437b-be94-3b0ee54546f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.815192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.815052    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrwlv\" (UniqueName: \"kubernetes.io/projected/1587b50a-ece0-43b5-a891-3e42831904d7-kube-api-access-lrwlv\") pod \"payload-processing-e2e-mt-65805bcc-5db76fd474-948lt\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") " pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.815192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.815138    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1587b50a-ece0-43b5-a891-3e42831904d7-plugins-config-volume\") pod \"payload-processing-e2e-mt-65805bcc-5db76fd474-948lt\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") " pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.815323 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.815224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clpp4\" (UniqueName: \"kubernetes.io/projected/9913f895-35b7-437b-be94-3b0ee54546f1-kube-api-access-clpp4\") pod \"payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.916133 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.916102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9913f895-35b7-437b-be94-3b0ee54546f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.916133 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.916138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrwlv\" (UniqueName: \"kubernetes.io/projected/1587b50a-ece0-43b5-a891-3e42831904d7-kube-api-access-lrwlv\") pod \"payload-processing-e2e-mt-65805bcc-5db76fd474-948lt\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") " pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.916560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.916181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1587b50a-ece0-43b5-a891-3e42831904d7-plugins-config-volume\") pod \"payload-processing-e2e-mt-65805bcc-5db76fd474-948lt\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") " pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.916560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.916241    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clpp4\" (UniqueName: \"kubernetes.io/projected/9913f895-35b7-437b-be94-3b0ee54546f1-kube-api-access-clpp4\") pod \"payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.916936 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.916916    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1587b50a-ece0-43b5-a891-3e42831904d7-plugins-config-volume\") pod \"payload-processing-e2e-mt-65805bcc-5db76fd474-948lt\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") " pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.916986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.916915    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9913f895-35b7-437b-be94-3b0ee54546f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.926824 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.926788    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrwlv\" (UniqueName: \"kubernetes.io/projected/1587b50a-ece0-43b5-a891-3e42831904d7-kube-api-access-lrwlv\") pod \"payload-processing-e2e-mt-65805bcc-5db76fd474-948lt\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") " pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:47.929477 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.929446    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clpp4\" (UniqueName: \"kubernetes.io/projected/9913f895-35b7-437b-be94-3b0ee54546f1-kube-api-access-clpp4\") pod \"payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:47.939377 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.939359    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"]
Aug 03 16:40:47.941331 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:40:47.941305    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42495b00_b56f_46da_8931_062be1e4a3b8.slice/crio-bdd0ce8b84b7a2a3351b0b55aa60837618de7c4acb37c7d05d0156ac67fe7df2 WatchSource:0}: Error finding container bdd0ce8b84b7a2a3351b0b55aa60837618de7c4acb37c7d05d0156ac67fe7df2: Status 404 returned error can't find the container with id bdd0ce8b84b7a2a3351b0b55aa60837618de7c4acb37c7d05d0156ac67fe7df2
Aug 03 16:40:47.942729 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.942713    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:40:47.986484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:47.986428    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:48.025656 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.025626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:48.181869 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.181843    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"]
Aug 03 16:40:48.184283 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:40:48.184253    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1587b50a_ece0_43b5_a891_3e42831904d7.slice/crio-57808971c95469d9d2661d202cce30b912e8b9e471301c627d5414375d29d337 WatchSource:0}: Error finding container 57808971c95469d9d2661d202cce30b912e8b9e471301c627d5414375d29d337: Status 404 returned error can't find the container with id 57808971c95469d9d2661d202cce30b912e8b9e471301c627d5414375d29d337
Aug 03 16:40:48.347842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.347811    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"]
Aug 03 16:40:48.348447 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:40:48.348414    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9913f895_35b7_437b_be94_3b0ee54546f1.slice/crio-621a024bb093e38ac980401e340d89256e7f7e9f18583db56e4de7049a5d905d WatchSource:0}: Error finding container 621a024bb093e38ac980401e340d89256e7f7e9f18583db56e4de7049a5d905d: Status 404 returned error can't find the container with id 621a024bb093e38ac980401e340d89256e7f7e9f18583db56e4de7049a5d905d
Aug 03 16:40:48.807245 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.807206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt" event={"ID":"1587b50a-ece0-43b5-a891-3e42831904d7","Type":"ContainerStarted","Data":"12c24095022dcab1a8f54031f713b09f19639d80910ec0bb4c91484f39c39e4c"}
Aug 03 16:40:48.807452 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.807275    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt" event={"ID":"1587b50a-ece0-43b5-a891-3e42831904d7","Type":"ContainerStarted","Data":"57808971c95469d9d2661d202cce30b912e8b9e471301c627d5414375d29d337"}
Aug 03 16:40:48.807452 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.807307    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:40:48.808831 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.808800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g" event={"ID":"9913f895-35b7-437b-be94-3b0ee54546f1","Type":"ContainerStarted","Data":"e262d3d5397f370cc475b3aaedb373c8f3ed5f7058d92ff883fd4792fe84417c"}
Aug 03 16:40:48.808968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.808855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g" event={"ID":"9913f895-35b7-437b-be94-3b0ee54546f1","Type":"ContainerStarted","Data":"621a024bb093e38ac980401e340d89256e7f7e9f18583db56e4de7049a5d905d"}
Aug 03 16:40:48.808968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.808942    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:48.810417 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.810388    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" event={"ID":"42495b00-b56f-46da-8931-062be1e4a3b8","Type":"ContainerStarted","Data":"cc988f83f9d1ba09695c54036af7357d59aff1a2a6cb8a98bf72b2da99de8780"}
Aug 03 16:40:48.810514 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.810424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" event={"ID":"42495b00-b56f-46da-8931-062be1e4a3b8","Type":"ContainerStarted","Data":"bdd0ce8b84b7a2a3351b0b55aa60837618de7c4acb37c7d05d0156ac67fe7df2"}
Aug 03 16:40:48.810514 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.810466    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:48.832386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.832338    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt" podStartSLOduration=1.832323685 podStartE2EDuration="1.832323685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:40:48.831219608 +0000 UTC m=+2208.380505928" watchObservedRunningTime="2026-08-03 16:40:48.832323685 +0000 UTC m=+2208.381609981"
Aug 03 16:40:48.856371 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.856324    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" podStartSLOduration=2.318179524 podStartE2EDuration="2.856307351s" totalImagesPullingTime="538.127827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:40:48.854322531 +0000 UTC m=+2208.403608849" watchObservedRunningTime="2026-08-03 16:40:48.856307351 +0000 UTC m=+2208.405593667"
Aug 03 16:40:48.881761 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:48.881714    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g" podStartSLOduration=1.881698794 podStartE2EDuration="1.881698794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:40:48.879797848 +0000 UTC m=+2208.429084177" watchObservedRunningTime="2026-08-03 16:40:48.881698794 +0000 UTC m=+2208.430985110"
Aug 03 16:40:54.828057 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:54.828029    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:40:59.824205 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:59.824175    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:40:59.824699 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:40:59.824254    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:41:01.213358 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:41:01.213326    2560 scope.go:117] "RemoveContainer" containerID="6f2098976d3048abd1fead580fab0936416f84946a856ef0cb3ed47d09a6a297"
Aug 03 16:41:01.222328 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:41:01.222307    2560 scope.go:117] "RemoveContainer" containerID="dba05c0766f6eec705e175cfca4750cc81c08c0bbfc000b48c9a0184d99dbc7f"
Aug 03 16:41:07.269733 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:41:07.269699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:41:10.795187 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:41:10.795154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:41:29.047540 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:41:29.047484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:00.692600 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.692566    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"]
Aug 03 16:42:00.696203 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.696184    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:00.698958 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.698935    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:42:00.703148 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.703116    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"]
Aug 03 16:42:00.768868 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.768829    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e5c335-0060-4fe6-b39b-4bf491139820-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf\" (UID: \"01e5c335-0060-4fe6-b39b-4bf491139820\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:00.869450 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.869416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e5c335-0060-4fe6-b39b-4bf491139820-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf\" (UID: \"01e5c335-0060-4fe6-b39b-4bf491139820\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:00.870138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:00.870117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e5c335-0060-4fe6-b39b-4bf491139820-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf\" (UID: \"01e5c335-0060-4fe6-b39b-4bf491139820\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:01.010840 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:01.010747    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:42:01.019082 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:01.019058    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:01.246921 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:01.246888    2560 scope.go:117] "RemoveContainer" containerID="093068416fd0f9ec459388f2624e5302adacd6209985ec6d5bbe7f3127798a59"
Aug 03 16:42:01.370835 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:01.370803    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"]
Aug 03 16:42:01.374155 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:42:01.374128    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e5c335_0060_4fe6_b39b_4bf491139820.slice/crio-9ea5bf653ba70406f7e5da2db6db086a9a82e5d4e1d315538b7ba6942b4bc599 WatchSource:0}: Error finding container 9ea5bf653ba70406f7e5da2db6db086a9a82e5d4e1d315538b7ba6942b4bc599: Status 404 returned error can't find the container with id 9ea5bf653ba70406f7e5da2db6db086a9a82e5d4e1d315538b7ba6942b4bc599
Aug 03 16:42:02.202465 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:02.202424    2560 generic.go:356] "Generic (PLEG): container finished" podID="01e5c335-0060-4fe6-b39b-4bf491139820" containerID="8275786ffd29044f92b137db3a5f60c5a481ba19c76517c800fd43d265207285" exitCode=0
Aug 03 16:42:02.202847 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:02.202471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf" event={"ID":"01e5c335-0060-4fe6-b39b-4bf491139820","Type":"ContainerDied","Data":"8275786ffd29044f92b137db3a5f60c5a481ba19c76517c800fd43d265207285"}
Aug 03 16:42:02.202847 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:02.202514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf" event={"ID":"01e5c335-0060-4fe6-b39b-4bf491139820","Type":"ContainerStarted","Data":"9ea5bf653ba70406f7e5da2db6db086a9a82e5d4e1d315538b7ba6942b4bc599"}
Aug 03 16:42:03.351081 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:03.351052    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:03.496275 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:03.496192    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e5c335-0060-4fe6-b39b-4bf491139820-maas-api-service-ca\") pod \"01e5c335-0060-4fe6-b39b-4bf491139820\" (UID: \"01e5c335-0060-4fe6-b39b-4bf491139820\") "
Aug 03 16:42:03.496648 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:03.496617    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01e5c335-0060-4fe6-b39b-4bf491139820-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01e5c335-0060-4fe6-b39b-4bf491139820" (UID: "01e5c335-0060-4fe6-b39b-4bf491139820"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:42:03.597015 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:03.596981    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01e5c335-0060-4fe6-b39b-4bf491139820-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:04.214584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:04.214550    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"
Aug 03 16:42:04.214584 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:04.214562    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf" event={"ID":"01e5c335-0060-4fe6-b39b-4bf491139820","Type":"ContainerDied","Data":"9ea5bf653ba70406f7e5da2db6db086a9a82e5d4e1d315538b7ba6942b4bc599"}
Aug 03 16:42:04.214803 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:04.214614    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ea5bf653ba70406f7e5da2db6db086a9a82e5d4e1d315538b7ba6942b4bc599"
Aug 03 16:42:16.089619 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.089496    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"]
Aug 03 16:42:16.090381 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.090346    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" podUID="42495b00-b56f-46da-8931-062be1e4a3b8" containerName="maas-api" containerID="cri-o://cc988f83f9d1ba09695c54036af7357d59aff1a2a6cb8a98bf72b2da99de8780" gracePeriod=30
Aug 03 16:42:16.112219 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.112192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:16.137174 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.137131    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"]
Aug 03 16:42:16.137358 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.137324    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt" podUID="1587b50a-ece0-43b5-a891-3e42831904d7" containerName="payload-processing" containerID="cri-o://12c24095022dcab1a8f54031f713b09f19639d80910ec0bb4c91484f39c39e4c" gracePeriod=30
Aug 03 16:42:16.146734 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.146706    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"]
Aug 03 16:42:16.146969 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.146902    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g" podUID="9913f895-35b7-437b-be94-3b0ee54546f1" containerName="payload-pre-processing" containerID="cri-o://e262d3d5397f370cc475b3aaedb373c8f3ed5f7058d92ff883fd4792fe84417c" gracePeriod=30
Aug 03 16:42:16.186388 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.186334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:16.252776 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.252736    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:16.277438 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.277385    2560 generic.go:356] "Generic (PLEG): container finished" podID="9913f895-35b7-437b-be94-3b0ee54546f1" containerID="e262d3d5397f370cc475b3aaedb373c8f3ed5f7058d92ff883fd4792fe84417c" exitCode=0
Aug 03 16:42:16.277646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.277473    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g" event={"ID":"9913f895-35b7-437b-be94-3b0ee54546f1","Type":"ContainerDied","Data":"e262d3d5397f370cc475b3aaedb373c8f3ed5f7058d92ff883fd4792fe84417c"}
Aug 03 16:42:16.279695 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.279661    2560 generic.go:356] "Generic (PLEG): container finished" podID="42495b00-b56f-46da-8931-062be1e4a3b8" containerID="cc988f83f9d1ba09695c54036af7357d59aff1a2a6cb8a98bf72b2da99de8780" exitCode=0
Aug 03 16:42:16.279887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.279729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" event={"ID":"42495b00-b56f-46da-8931-062be1e4a3b8","Type":"ContainerDied","Data":"cc988f83f9d1ba09695c54036af7357d59aff1a2a6cb8a98bf72b2da99de8780"}
Aug 03 16:42:16.281874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.281848    2560 generic.go:356] "Generic (PLEG): container finished" podID="1587b50a-ece0-43b5-a891-3e42831904d7" containerID="12c24095022dcab1a8f54031f713b09f19639d80910ec0bb4c91484f39c39e4c" exitCode=0
Aug 03 16:42:16.281986 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.281898    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt" event={"ID":"1587b50a-ece0-43b5-a891-3e42831904d7","Type":"ContainerDied","Data":"12c24095022dcab1a8f54031f713b09f19639d80910ec0bb4c91484f39c39e4c"}
Aug 03 16:42:16.317921 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.317879    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"]
Aug 03 16:42:16.319171 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.319146    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:16.321776 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.321732    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-65805bcc-4lqzf"]
Aug 03 16:42:16.459209 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.459184    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:42:16.462132 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.462110    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:42:16.465714 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.465693    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:42:16.615823 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.615782    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1587b50a-ece0-43b5-a891-3e42831904d7-plugins-config-volume\") pod \"1587b50a-ece0-43b5-a891-3e42831904d7\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") "
Aug 03 16:42:16.616005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.615869    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpbml\" (UniqueName: \"kubernetes.io/projected/42495b00-b56f-46da-8931-062be1e4a3b8-kube-api-access-kpbml\") pod \"42495b00-b56f-46da-8931-062be1e4a3b8\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") "
Aug 03 16:42:16.616005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.615908    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9913f895-35b7-437b-be94-3b0ee54546f1-plugins-config-volume\") pod \"9913f895-35b7-437b-be94-3b0ee54546f1\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") "
Aug 03 16:42:16.616005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.615931    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42495b00-b56f-46da-8931-062be1e4a3b8-openshift-service-ca\") pod \"42495b00-b56f-46da-8931-062be1e4a3b8\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") "
Aug 03 16:42:16.616005 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.615956    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clpp4\" (UniqueName: \"kubernetes.io/projected/9913f895-35b7-437b-be94-3b0ee54546f1-kube-api-access-clpp4\") pod \"9913f895-35b7-437b-be94-3b0ee54546f1\" (UID: \"9913f895-35b7-437b-be94-3b0ee54546f1\") "
Aug 03 16:42:16.616239 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.616021    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls\") pod \"42495b00-b56f-46da-8931-062be1e4a3b8\" (UID: \"42495b00-b56f-46da-8931-062be1e4a3b8\") "
Aug 03 16:42:16.616239 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.616071    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrwlv\" (UniqueName: \"kubernetes.io/projected/1587b50a-ece0-43b5-a891-3e42831904d7-kube-api-access-lrwlv\") pod \"1587b50a-ece0-43b5-a891-3e42831904d7\" (UID: \"1587b50a-ece0-43b5-a891-3e42831904d7\") "
Aug 03 16:42:16.616349 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.616325    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42495b00-b56f-46da-8931-062be1e4a3b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42495b00-b56f-46da-8931-062be1e4a3b8" (UID: "42495b00-b56f-46da-8931-062be1e4a3b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:42:16.616407 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.616317    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1587b50a-ece0-43b5-a891-3e42831904d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1587b50a-ece0-43b5-a891-3e42831904d7" (UID: "1587b50a-ece0-43b5-a891-3e42831904d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:42:16.616407 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.616350    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9913f895-35b7-437b-be94-3b0ee54546f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9913f895-35b7-437b-be94-3b0ee54546f1" (UID: "9913f895-35b7-437b-be94-3b0ee54546f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:42:16.618305 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.618230    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42495b00-b56f-46da-8931-062be1e4a3b8" (UID: "42495b00-b56f-46da-8931-062be1e4a3b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:42:16.618305 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.618260    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9913f895-35b7-437b-be94-3b0ee54546f1-kube-api-access-clpp4" (OuterVolumeSpecName: "kube-api-access-clpp4") pod "9913f895-35b7-437b-be94-3b0ee54546f1" (UID: "9913f895-35b7-437b-be94-3b0ee54546f1"). InnerVolumeSpecName "kube-api-access-clpp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:42:16.618305 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.618232    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42495b00-b56f-46da-8931-062be1e4a3b8-kube-api-access-kpbml" (OuterVolumeSpecName: "kube-api-access-kpbml") pod "42495b00-b56f-46da-8931-062be1e4a3b8" (UID: "42495b00-b56f-46da-8931-062be1e4a3b8"). InnerVolumeSpecName "kube-api-access-kpbml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:42:16.618305 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.618297    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1587b50a-ece0-43b5-a891-3e42831904d7-kube-api-access-lrwlv" (OuterVolumeSpecName: "kube-api-access-lrwlv") pod "1587b50a-ece0-43b5-a891-3e42831904d7" (UID: "1587b50a-ece0-43b5-a891-3e42831904d7"). InnerVolumeSpecName "kube-api-access-lrwlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:42:16.716838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716794    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9913f895-35b7-437b-be94-3b0ee54546f1-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.716838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716831    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42495b00-b56f-46da-8931-062be1e4a3b8-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.716838 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716841    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clpp4\" (UniqueName: \"kubernetes.io/projected/9913f895-35b7-437b-be94-3b0ee54546f1-kube-api-access-clpp4\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.717074 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716851    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42495b00-b56f-46da-8931-062be1e4a3b8-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.717074 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716860    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrwlv\" (UniqueName: \"kubernetes.io/projected/1587b50a-ece0-43b5-a891-3e42831904d7-kube-api-access-lrwlv\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.717074 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716868    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1587b50a-ece0-43b5-a891-3e42831904d7-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.717074 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.716876    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpbml\" (UniqueName: \"kubernetes.io/projected/42495b00-b56f-46da-8931-062be1e4a3b8-kube-api-access-kpbml\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:42:16.987188 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:16.987103    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01e5c335-0060-4fe6-b39b-4bf491139820" path="/var/lib/kubelet/pods/01e5c335-0060-4fe6-b39b-4bf491139820/volumes"
Aug 03 16:42:17.287951 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.287923    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"
Aug 03 16:42:17.288386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.287930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt" event={"ID":"1587b50a-ece0-43b5-a891-3e42831904d7","Type":"ContainerDied","Data":"57808971c95469d9d2661d202cce30b912e8b9e471301c627d5414375d29d337"}
Aug 03 16:42:17.288386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.288066    2560 scope.go:117] "RemoveContainer" containerID="12c24095022dcab1a8f54031f713b09f19639d80910ec0bb4c91484f39c39e4c"
Aug 03 16:42:17.289496 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.289343    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"
Aug 03 16:42:17.289496 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.289368    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g" event={"ID":"9913f895-35b7-437b-be94-3b0ee54546f1","Type":"ContainerDied","Data":"621a024bb093e38ac980401e340d89256e7f7e9f18583db56e4de7049a5d905d"}
Aug 03 16:42:17.290990 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.290969    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm" event={"ID":"42495b00-b56f-46da-8931-062be1e4a3b8","Type":"ContainerDied","Data":"bdd0ce8b84b7a2a3351b0b55aa60837618de7c4acb37c7d05d0156ac67fe7df2"}
Aug 03 16:42:17.291085 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.290993    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"
Aug 03 16:42:17.299757 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.299738    2560 scope.go:117] "RemoveContainer" containerID="e262d3d5397f370cc475b3aaedb373c8f3ed5f7058d92ff883fd4792fe84417c"
Aug 03 16:42:17.307763 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.307747    2560 scope.go:117] "RemoveContainer" containerID="cc988f83f9d1ba09695c54036af7357d59aff1a2a6cb8a98bf72b2da99de8780"
Aug 03 16:42:17.315413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.315329    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"]
Aug 03 16:42:17.317957 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.317939    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65805bcc-7b97b54d7d-57bhm"]
Aug 03 16:42:17.332683 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.332661    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"]
Aug 03 16:42:17.337494 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.337472    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-65805bcc-5db76fd474-948lt"]
Aug 03 16:42:17.348768 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.348748    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"]
Aug 03 16:42:17.352825 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:17.352807    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-65805bcc-64445bdd7b-tg72g"]
Aug 03 16:42:18.982320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:18.982287    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:18.987409 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:18.987382    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1587b50a-ece0-43b5-a891-3e42831904d7" path="/var/lib/kubelet/pods/1587b50a-ece0-43b5-a891-3e42831904d7/volumes"
Aug 03 16:42:18.987819 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:18.987805    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42495b00-b56f-46da-8931-062be1e4a3b8" path="/var/lib/kubelet/pods/42495b00-b56f-46da-8931-062be1e4a3b8/volumes"
Aug 03 16:42:18.988137 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:18.988126    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9913f895-35b7-437b-be94-3b0ee54546f1" path="/var/lib/kubelet/pods/9913f895-35b7-437b-be94-3b0ee54546f1/volumes"
Aug 03 16:42:19.593207 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:19.593169    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:42:24.672898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:42:24.672863    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:02.018167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:02.018133    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:03.235255 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:03.235217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:08.113513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:08.113479    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:13.997834 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:13.997787    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:14.060013 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:14.059983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:14.306957 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:14.306922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:14.535494 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:14.535426    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:25.942476 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:25.942431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:26.041201 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:26.041166    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:26.142376 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:26.142345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:26.841646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:26.841601    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:26.940134 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:26.940095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:27.040201 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:27.040167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:28.400020 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:28.399989    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:35.134108 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:35.134070    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:35.204064 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:35.204035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:35.824174 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:35.824141    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:42.053046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:42.053014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:42.119463 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:42.119432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:42.234159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:42.234122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:42.334437 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:42.334340    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:52.805390 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:52.805357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:43:53.054006 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:43:53.053967    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:01.096540 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:01.096501    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:44:01.105537 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:01.105501    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:44:14.047260 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:14.047226    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:20.113153 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:20.113118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:20.215649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:20.215619    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:20.487079 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:20.486997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:20.820474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:20.820443    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:21.102772 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:21.102702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:24.110253 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:24.110131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:24.202489 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:24.202458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:25.201275 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201238    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"]
Aug 03 16:44:25.201809 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201757    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9913f895-35b7-437b-be94-3b0ee54546f1" containerName="payload-pre-processing"
Aug 03 16:44:25.201809 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201773    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9913f895-35b7-437b-be94-3b0ee54546f1" containerName="payload-pre-processing"
Aug 03 16:44:25.201809 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201805    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42495b00-b56f-46da-8931-062be1e4a3b8" containerName="maas-api"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201814    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="42495b00-b56f-46da-8931-062be1e4a3b8" containerName="maas-api"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201834    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1587b50a-ece0-43b5-a891-3e42831904d7" containerName="payload-processing"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201843    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1587b50a-ece0-43b5-a891-3e42831904d7" containerName="payload-processing"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201853    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e5c335-0060-4fe6-b39b-4bf491139820" containerName="revoke-keys"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201861    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e5c335-0060-4fe6-b39b-4bf491139820" containerName="revoke-keys"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201946    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="42495b00-b56f-46da-8931-062be1e4a3b8" containerName="maas-api"
Aug 03 16:44:25.201954 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201954    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9913f895-35b7-437b-be94-3b0ee54546f1" containerName="payload-pre-processing"
Aug 03 16:44:25.202169 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201960    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="01e5c335-0060-4fe6-b39b-4bf491139820" containerName="revoke-keys"
Aug 03 16:44:25.202169 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.201967    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1587b50a-ece0-43b5-a891-3e42831904d7" containerName="payload-processing"
Aug 03 16:44:25.205274 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.205256    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.207975 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.207954    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7ca182-serving-cert\""
Aug 03 16:44:25.212417 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.212394    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"]
Aug 03 16:44:25.224059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.224038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:25.254021 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.253988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.254178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.254045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88704101-31d9-4555-88b9-3449c539e2c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.254178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.254116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmd66\" (UniqueName: \"kubernetes.io/projected/88704101-31d9-4555-88b9-3449c539e2c9-kube-api-access-hmd66\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.355232 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.355199    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.355432 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.355248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88704101-31d9-4555-88b9-3449c539e2c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.355432 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.355278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmd66\" (UniqueName: \"kubernetes.io/projected/88704101-31d9-4555-88b9-3449c539e2c9-kube-api-access-hmd66\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.355432 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:44:25.355343    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-serving-cert: secret "maas-api-e2e-inf-a-7ca182-serving-cert" not found
Aug 03 16:44:25.355432 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:44:25.355407    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls podName:88704101-31d9-4555-88b9-3449c539e2c9 nodeName:}" failed. No retries permitted until 2026-08-03 16:44:25.855390515 +0000 UTC m=+2425.404676812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls") pod "maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" (UID: "88704101-31d9-4555-88b9-3449c539e2c9") : secret "maas-api-e2e-inf-a-7ca182-serving-cert" not found
Aug 03 16:44:25.356019 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.355995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88704101-31d9-4555-88b9-3449c539e2c9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.365064 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.365044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmd66\" (UniqueName: \"kubernetes.io/projected/88704101-31d9-4555-88b9-3449c539e2c9-kube-api-access-hmd66\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.464698 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.464619    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:25.783211 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.783186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:25.824900 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.824866    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"]
Aug 03 16:44:25.828592 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.828572    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:25.831374 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.831355    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7ca182\""
Aug 03 16:44:25.831478 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.831418    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7ca182-dockercfg-znhsh\""
Aug 03 16:44:25.840114 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.840091    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"]
Aug 03 16:44:25.856563 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.856512    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"]
Aug 03 16:44:25.860054 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.860034    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:25.860349 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.860327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:25.860404 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.860387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9vjv\" (UniqueName: \"kubernetes.io/projected/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-kube-api-access-r9vjv\") pod \"payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:25.860451 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.860435    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.863130 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.863105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls\") pod \"maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:25.870065 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.870038    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"]
Aug 03 16:44:25.874734 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.874704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:25.961202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.961164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7xkl\" (UniqueName: \"kubernetes.io/projected/4465e57d-0d08-4a48-9916-84e62d1dcfdd-kube-api-access-f7xkl\") pod \"payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:25.961410 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.961229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:25.961410 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.961301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9vjv\" (UniqueName: \"kubernetes.io/projected/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-kube-api-access-r9vjv\") pod \"payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:25.961410 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.961357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465e57d-0d08-4a48-9916-84e62d1dcfdd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:25.961898 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.961879    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:25.970515 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:25.970485    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9vjv\" (UniqueName: \"kubernetes.io/projected/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-kube-api-access-r9vjv\") pod \"payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:26.062361 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.062251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465e57d-0d08-4a48-9916-84e62d1dcfdd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:26.062361 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.062311    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7xkl\" (UniqueName: \"kubernetes.io/projected/4465e57d-0d08-4a48-9916-84e62d1dcfdd-kube-api-access-f7xkl\") pod \"payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:26.063038 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.063018    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465e57d-0d08-4a48-9916-84e62d1dcfdd-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:26.070587 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.070560    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7xkl\" (UniqueName: \"kubernetes.io/projected/4465e57d-0d08-4a48-9916-84e62d1dcfdd-kube-api-access-f7xkl\") pod \"payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:26.117320 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.117282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:26.140783 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.140758    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:26.182071 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.181783    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:26.295884 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.295126    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"]
Aug 03 16:44:26.298897 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:44:26.298841    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88704101_31d9_4555_88b9_3449c539e2c9.slice/crio-349123cae1d80fd3ae1116bf92656a16d7d96cc0e9a6877626c2c487e64275da WatchSource:0}: Error finding container 349123cae1d80fd3ae1116bf92656a16d7d96cc0e9a6877626c2c487e64275da: Status 404 returned error can't find the container with id 349123cae1d80fd3ae1116bf92656a16d7d96cc0e9a6877626c2c487e64275da
Aug 03 16:44:26.535681 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.535650    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"]
Aug 03 16:44:26.537863 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:44:26.537831    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacae0d3c_8eef_4aa3_acca_4089f5fde0f0.slice/crio-1e6a7c7c910609bb11044a7ae335ea2063257744fd55b40d3ec47e59d19d05c2 WatchSource:0}: Error finding container 1e6a7c7c910609bb11044a7ae335ea2063257744fd55b40d3ec47e59d19d05c2: Status 404 returned error can't find the container with id 1e6a7c7c910609bb11044a7ae335ea2063257744fd55b40d3ec47e59d19d05c2
Aug 03 16:44:26.573820 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.573451    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"]
Aug 03 16:44:26.579119 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:44:26.579064    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4465e57d_0d08_4a48_9916_84e62d1dcfdd.slice/crio-09ba0e14c05f128f27761ea3381943fd306399a5614a9d114340ceda9a4d18d9 WatchSource:0}: Error finding container 09ba0e14c05f128f27761ea3381943fd306399a5614a9d114340ceda9a4d18d9: Status 404 returned error can't find the container with id 09ba0e14c05f128f27761ea3381943fd306399a5614a9d114340ceda9a4d18d9
Aug 03 16:44:26.929031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.928984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw" event={"ID":"4465e57d-0d08-4a48-9916-84e62d1dcfdd","Type":"ContainerStarted","Data":"558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7"}
Aug 03 16:44:26.929238 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.929048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw" event={"ID":"4465e57d-0d08-4a48-9916-84e62d1dcfdd","Type":"ContainerStarted","Data":"09ba0e14c05f128f27761ea3381943fd306399a5614a9d114340ceda9a4d18d9"}
Aug 03 16:44:26.929238 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.929151    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:26.930985 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.930958    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr" event={"ID":"acae0d3c-8eef-4aa3-acca-4089f5fde0f0","Type":"ContainerStarted","Data":"0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c"}
Aug 03 16:44:26.931138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.931014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr" event={"ID":"acae0d3c-8eef-4aa3-acca-4089f5fde0f0","Type":"ContainerStarted","Data":"1e6a7c7c910609bb11044a7ae335ea2063257744fd55b40d3ec47e59d19d05c2"}
Aug 03 16:44:26.931138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.931042    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:26.932497 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.932469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" event={"ID":"88704101-31d9-4555-88b9-3449c539e2c9","Type":"ContainerStarted","Data":"1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a"}
Aug 03 16:44:26.932647 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.932510    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" event={"ID":"88704101-31d9-4555-88b9-3449c539e2c9","Type":"ContainerStarted","Data":"349123cae1d80fd3ae1116bf92656a16d7d96cc0e9a6877626c2c487e64275da"}
Aug 03 16:44:26.932647 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.932618    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:26.946781 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.946726    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw" podStartSLOduration=1.9467098790000001 podStartE2EDuration="1.946709879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:26.946433067 +0000 UTC m=+2426.495719406" watchObservedRunningTime="2026-08-03 16:44:26.946709879 +0000 UTC m=+2426.495996195"
Aug 03 16:44:26.966673 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.966622    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" podStartSLOduration=1.476311949 podStartE2EDuration="1.966607315s" totalImagesPullingTime="490.295366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:26.964924277 +0000 UTC m=+2426.514210609" watchObservedRunningTime="2026-08-03 16:44:26.966607315 +0000 UTC m=+2426.515893631"
Aug 03 16:44:26.982057 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:26.982005    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr" podStartSLOduration=1.9819868 podStartE2EDuration="1.9819868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:26.981064041 +0000 UTC m=+2426.530350357" watchObservedRunningTime="2026-08-03 16:44:26.9819868 +0000 UTC m=+2426.531273118"
Aug 03 16:44:32.943823 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:32.943792    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:44:37.939457 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:37.939426    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:44:37.939885 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:37.939746    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:44:49.052329 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:49.052297    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:49.397083 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:49.397002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:49.705426 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:49.705345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:49.984448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:49.984368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:52.164682 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:52.164644    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:52.256740 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:52.256705    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:53.227103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.227073    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"]
Aug 03 16:44:53.230749 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.230728    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.233164 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.233141    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d7cee2-serving-cert\""
Aug 03 16:44:53.244086 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.244059    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"]
Aug 03 16:44:53.265099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.265072    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:53.312248 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.312215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg8nz\" (UniqueName: \"kubernetes.io/projected/f17876e5-e23b-4d92-bdde-01a5d259a667-kube-api-access-zg8nz\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.312416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.312256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f17876e5-e23b-4d92-bdde-01a5d259a667-maas-api-tls\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.312416 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.312280    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17876e5-e23b-4d92-bdde-01a5d259a667-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.368836 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.368800    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:53.413228 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.413196    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg8nz\" (UniqueName: \"kubernetes.io/projected/f17876e5-e23b-4d92-bdde-01a5d259a667-kube-api-access-zg8nz\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.413421 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.413242    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f17876e5-e23b-4d92-bdde-01a5d259a667-maas-api-tls\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.413421 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.413264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17876e5-e23b-4d92-bdde-01a5d259a667-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.413963 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.413941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17876e5-e23b-4d92-bdde-01a5d259a667-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.415649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.415632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f17876e5-e23b-4d92-bdde-01a5d259a667-maas-api-tls\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.423937 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.423917    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg8nz\" (UniqueName: \"kubernetes.io/projected/f17876e5-e23b-4d92-bdde-01a5d259a667-kube-api-access-zg8nz\") pod \"maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.545444 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.545401    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:53.686125 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.686093    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"]
Aug 03 16:44:53.688846 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:44:53.688821    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17876e5_e23b_4d92_bdde_01a5d259a667.slice/crio-e5310e31a5dd33542082634c7e0fe99c1f2c836852f8e313dab5d793e7e1fc5c WatchSource:0}: Error finding container e5310e31a5dd33542082634c7e0fe99c1f2c836852f8e313dab5d793e7e1fc5c: Status 404 returned error can't find the container with id e5310e31a5dd33542082634c7e0fe99c1f2c836852f8e313dab5d793e7e1fc5c
Aug 03 16:44:53.766914 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.766879    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:53.869968 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.869887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:44:53.934646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.934610    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"]
Aug 03 16:44:53.939325 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.939305    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:53.942966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.942947    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d7cee2\""
Aug 03 16:44:53.943103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.943043    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d7cee2-dockercfg-bvmk7\""
Aug 03 16:44:53.945070 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.945049    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"]
Aug 03 16:44:53.948694 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.948677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:53.966424 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.966402    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"]
Aug 03 16:44:53.968618 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:53.968596    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"]
Aug 03 16:44:54.019517 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.019483    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd4zt\" (UniqueName: \"kubernetes.io/projected/75f580d1-910b-491a-b8c1-237ffb1ad047-kube-api-access-cd4zt\") pod \"payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.019725 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.019554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75f580d1-910b-491a-b8c1-237ffb1ad047-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.019725 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.019596    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4284b30-fb0a-4f4f-b339-53cb5b73635d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.019725 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.019613    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb662\" (UniqueName: \"kubernetes.io/projected/d4284b30-fb0a-4f4f-b339-53cb5b73635d-kube-api-access-pb662\") pod \"payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.073971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.073942    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq" event={"ID":"f17876e5-e23b-4d92-bdde-01a5d259a667","Type":"ContainerStarted","Data":"e5310e31a5dd33542082634c7e0fe99c1f2c836852f8e313dab5d793e7e1fc5c"}
Aug 03 16:44:54.120247 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.120185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd4zt\" (UniqueName: \"kubernetes.io/projected/75f580d1-910b-491a-b8c1-237ffb1ad047-kube-api-access-cd4zt\") pod \"payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.120372 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.120261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75f580d1-910b-491a-b8c1-237ffb1ad047-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.120580 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.120555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4284b30-fb0a-4f4f-b339-53cb5b73635d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.120682 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.120597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb662\" (UniqueName: \"kubernetes.io/projected/d4284b30-fb0a-4f4f-b339-53cb5b73635d-kube-api-access-pb662\") pod \"payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.120989 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.120966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75f580d1-910b-491a-b8c1-237ffb1ad047-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.121186 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.121166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4284b30-fb0a-4f4f-b339-53cb5b73635d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.131742 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.131723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb662\" (UniqueName: \"kubernetes.io/projected/d4284b30-fb0a-4f4f-b339-53cb5b73635d-kube-api-access-pb662\") pod \"payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.131824 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.131771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd4zt\" (UniqueName: \"kubernetes.io/projected/75f580d1-910b-491a-b8c1-237ffb1ad047-kube-api-access-cd4zt\") pod \"payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.251138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.251101    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:54.259057 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.259034    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:54.405902 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.405855    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"]
Aug 03 16:44:54.408187 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:44:54.408149    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f580d1_910b_491a_b8c1_237ffb1ad047.slice/crio-5e738d7b8322a6e21fafa5fec4e289b7ca5d2849326d9bfed0d1e9db06ff64c9 WatchSource:0}: Error finding container 5e738d7b8322a6e21fafa5fec4e289b7ca5d2849326d9bfed0d1e9db06ff64c9: Status 404 returned error can't find the container with id 5e738d7b8322a6e21fafa5fec4e289b7ca5d2849326d9bfed0d1e9db06ff64c9
Aug 03 16:44:54.435308 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:54.435280    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"]
Aug 03 16:44:54.437180 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:44:54.437156    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4284b30_fb0a_4f4f_b339_53cb5b73635d.slice/crio-62d17d5c3518682e27c60743f42c1552426536580f63f968643612abf3f9319d WatchSource:0}: Error finding container 62d17d5c3518682e27c60743f42c1552426536580f63f968643612abf3f9319d: Status 404 returned error can't find the container with id 62d17d5c3518682e27c60743f42c1552426536580f63f968643612abf3f9319d
Aug 03 16:44:55.080224 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.080184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq" event={"ID":"f17876e5-e23b-4d92-bdde-01a5d259a667","Type":"ContainerStarted","Data":"d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f"}
Aug 03 16:44:55.080415 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.080253    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:44:55.081564 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.081520    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr" event={"ID":"d4284b30-fb0a-4f4f-b339-53cb5b73635d","Type":"ContainerStarted","Data":"cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123"}
Aug 03 16:44:55.081687 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.081583    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr" event={"ID":"d4284b30-fb0a-4f4f-b339-53cb5b73635d","Type":"ContainerStarted","Data":"62d17d5c3518682e27c60743f42c1552426536580f63f968643612abf3f9319d"}
Aug 03 16:44:55.081687 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.081611    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:44:55.083031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.083012    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg" event={"ID":"75f580d1-910b-491a-b8c1-237ffb1ad047","Type":"ContainerStarted","Data":"90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de"}
Aug 03 16:44:55.083102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.083049    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg" event={"ID":"75f580d1-910b-491a-b8c1-237ffb1ad047","Type":"ContainerStarted","Data":"5e738d7b8322a6e21fafa5fec4e289b7ca5d2849326d9bfed0d1e9db06ff64c9"}
Aug 03 16:44:55.083102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.083070    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:44:55.099810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.099753    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq" podStartSLOduration=1.679174661 podStartE2EDuration="2.099733613s" totalImagesPullingTime="420.558952ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:55.096972317 +0000 UTC m=+2454.646258646" watchObservedRunningTime="2026-08-03 16:44:55.099733613 +0000 UTC m=+2454.649019929"
Aug 03 16:44:55.115130 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.115084    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr" podStartSLOduration=2.115069619 podStartE2EDuration="2.115069619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:55.112820449 +0000 UTC m=+2454.662106765" watchObservedRunningTime="2026-08-03 16:44:55.115069619 +0000 UTC m=+2454.664355934"
Aug 03 16:44:55.128795 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:44:55.128750    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg" podStartSLOduration=2.1287344 podStartE2EDuration="2.1287344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:55.1282592 +0000 UTC m=+2454.677545528" watchObservedRunningTime="2026-08-03 16:44:55.1287344 +0000 UTC m=+2454.678020719"
Aug 03 16:45:00.141422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.141388    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"]
Aug 03 16:45:00.146245 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.146221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"
Aug 03 16:45:00.148981 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.148958    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:45:00.154471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.154442    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"]
Aug 03 16:45:00.159065 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.159045    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"
Aug 03 16:45:00.502418 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.502373    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"]
Aug 03 16:45:00.504785 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:45:00.504753    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38679053_2e5d_41a2_ba8b_66b41d602d29.slice/crio-0ed974b378009d92b9211f5131368535a754085ce4e769bceca287c8d1d56a9a WatchSource:0}: Error finding container 0ed974b378009d92b9211f5131368535a754085ce4e769bceca287c8d1d56a9a: Status 404 returned error can't find the container with id 0ed974b378009d92b9211f5131368535a754085ce4e769bceca287c8d1d56a9a
Aug 03 16:45:00.838143 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:00.838115    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:45:01.094141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:01.094069    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:45:01.115130 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:01.115099    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n" event={"ID":"38679053-2e5d-41a2-ba8b-66b41d602d29","Type":"ContainerStarted","Data":"8c0547ce0e2d16d5052adda4831debf5c7e6792273054707847fd6c8990adaef"}
Aug 03 16:45:01.115284 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:01.115149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n" event={"ID":"38679053-2e5d-41a2-ba8b-66b41d602d29","Type":"ContainerStarted","Data":"0ed974b378009d92b9211f5131368535a754085ce4e769bceca287c8d1d56a9a"}
Aug 03 16:45:01.143833 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:01.143767    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n" podStartSLOduration=1.143740725 podStartE2EDuration="1.143740725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:45:01.14063815 +0000 UTC m=+2460.689924476" watchObservedRunningTime="2026-08-03 16:45:01.143740725 +0000 UTC m=+2460.693027041"
Aug 03 16:45:01.298935 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:01.298893    2560 scope.go:117] "RemoveContainer" containerID="e0c3d063ef62905defcc76d1dc0d716f16ea9d807756104aaf5ddbe2f1fb237a"
Aug 03 16:45:02.121790 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:02.121753    2560 generic.go:356] "Generic (PLEG): container finished" podID="38679053-2e5d-41a2-ba8b-66b41d602d29" containerID="8c0547ce0e2d16d5052adda4831debf5c7e6792273054707847fd6c8990adaef" exitCode=0
Aug 03 16:45:02.121962 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:02.121843    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n" event={"ID":"38679053-2e5d-41a2-ba8b-66b41d602d29","Type":"ContainerDied","Data":"8c0547ce0e2d16d5052adda4831debf5c7e6792273054707847fd6c8990adaef"}
Aug 03 16:45:03.259087 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:03.259061    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"
Aug 03 16:45:04.133707 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:04.133669    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n" event={"ID":"38679053-2e5d-41a2-ba8b-66b41d602d29","Type":"ContainerDied","Data":"0ed974b378009d92b9211f5131368535a754085ce4e769bceca287c8d1d56a9a"}
Aug 03 16:45:04.133707 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:04.133700    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"
Aug 03 16:45:04.133912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:04.133717    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ed974b378009d92b9211f5131368535a754085ce4e769bceca287c8d1d56a9a"
Aug 03 16:45:06.090720 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:06.090689    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:45:06.091135 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:06.090765    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:45:17.482468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:17.482438    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:45:19.254197 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:19.254166    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:45:50.993983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:45:50.993953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:46:09.477386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:09.477304    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:46:52.573163 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:52.573119    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:46:53.459134 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.459098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:46:53.586054 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.586018    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"]
Aug 03 16:46:53.586589 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.586460    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38679053-2e5d-41a2-ba8b-66b41d602d29" containerName="cleanup"
Aug 03 16:46:53.586589 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.586472    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38679053-2e5d-41a2-ba8b-66b41d602d29" containerName="cleanup"
Aug 03 16:46:53.586589 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.586556    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38679053-2e5d-41a2-ba8b-66b41d602d29" containerName="cleanup"
Aug 03 16:46:53.589703 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.589682    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:53.592447 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.592429    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:46:53.598746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.598095    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"]
Aug 03 16:46:53.675843 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.675814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15120425-d678-472e-b08c-d9f86038c666-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx\" (UID: \"15120425-d678-472e-b08c-d9f86038c666\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:53.776600 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.776496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15120425-d678-472e-b08c-d9f86038c666-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx\" (UID: \"15120425-d678-472e-b08c-d9f86038c666\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:53.777178 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.777152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15120425-d678-472e-b08c-d9f86038c666-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx\" (UID: \"15120425-d678-472e-b08c-d9f86038c666\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:53.903938 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:53.903905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:54.046177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:54.045207    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"]
Aug 03 16:46:54.054828 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:46:54.054786    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15120425_d678_472e_b08c_d9f86038c666.slice/crio-e5ef34fc83705add4b7afad16ff60573037fdd0083f81d91cb1536875659ae32 WatchSource:0}: Error finding container e5ef34fc83705add4b7afad16ff60573037fdd0083f81d91cb1536875659ae32: Status 404 returned error can't find the container with id e5ef34fc83705add4b7afad16ff60573037fdd0083f81d91cb1536875659ae32
Aug 03 16:46:54.697551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:54.697443    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx" event={"ID":"15120425-d678-472e-b08c-d9f86038c666","Type":"ContainerStarted","Data":"7cbc6ffd685d804f412d86215058c683b15df15d10bfe2026574098b5722cf64"}
Aug 03 16:46:54.697551 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:54.697497    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx" event={"ID":"15120425-d678-472e-b08c-d9f86038c666","Type":"ContainerStarted","Data":"e5ef34fc83705add4b7afad16ff60573037fdd0083f81d91cb1536875659ae32"}
Aug 03 16:46:54.713941 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:54.713890    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx" podStartSLOduration=1.71387516 podStartE2EDuration="1.71387516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:46:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:46:54.712523452 +0000 UTC m=+2574.261809781" watchObservedRunningTime="2026-08-03 16:46:54.71387516 +0000 UTC m=+2574.263161480"
Aug 03 16:46:55.703938 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:55.703899    2560 generic.go:356] "Generic (PLEG): container finished" podID="15120425-d678-472e-b08c-d9f86038c666" containerID="7cbc6ffd685d804f412d86215058c683b15df15d10bfe2026574098b5722cf64" exitCode=0
Aug 03 16:46:55.704336 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:55.703980    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx" event={"ID":"15120425-d678-472e-b08c-d9f86038c666","Type":"ContainerDied","Data":"7cbc6ffd685d804f412d86215058c683b15df15d10bfe2026574098b5722cf64"}
Aug 03 16:46:56.851323 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:56.851299    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:56.901373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:56.901342    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15120425-d678-472e-b08c-d9f86038c666-maas-api-service-ca\") pod \"15120425-d678-472e-b08c-d9f86038c666\" (UID: \"15120425-d678-472e-b08c-d9f86038c666\") "
Aug 03 16:46:56.901702 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:56.901681    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15120425-d678-472e-b08c-d9f86038c666-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15120425-d678-472e-b08c-d9f86038c666" (UID: "15120425-d678-472e-b08c-d9f86038c666"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:46:57.002697 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:57.002666    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15120425-d678-472e-b08c-d9f86038c666-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:46:57.715487 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:57.715450    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx" event={"ID":"15120425-d678-472e-b08c-d9f86038c666","Type":"ContainerDied","Data":"e5ef34fc83705add4b7afad16ff60573037fdd0083f81d91cb1536875659ae32"}
Aug 03 16:46:57.715487 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:57.715480    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"
Aug 03 16:46:57.715715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:46:57.715502    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5ef34fc83705add4b7afad16ff60573037fdd0083f81d91cb1536875659ae32"
Aug 03 16:47:04.077660 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.077622    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"]
Aug 03 16:47:04.078134 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.077900    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" podUID="88704101-31d9-4555-88b9-3449c539e2c9" containerName="maas-api" containerID="cri-o://1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a" gracePeriod=30
Aug 03 16:47:04.103970 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.103939    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:04.104075 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.103996    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"]
Aug 03 16:47:04.107962 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.107940    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7ca182-29762925-cpv6n"]
Aug 03 16:47:04.132471 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.132446    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"]
Aug 03 16:47:04.132671 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.132645    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw" podUID="4465e57d-0d08-4a48-9916-84e62d1dcfdd" containerName="payload-processing" containerID="cri-o://558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7" gracePeriod=30
Aug 03 16:47:04.139190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.139167    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"]
Aug 03 16:47:04.139405 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.139381    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr" podUID="acae0d3c-8eef-4aa3-acca-4089f5fde0f0" containerName="payload-pre-processing" containerID="cri-o://0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c" gracePeriod=30
Aug 03 16:47:04.193288 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.193256    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:04.281770 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.281718    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"]
Aug 03 16:47:04.286306 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.286198    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ca182-fdjwx"]
Aug 03 16:47:04.291627 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.291175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:04.443035 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.443005    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:47:04.445963 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.445945    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:47:04.447468 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.447445    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:04.456362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.456340    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:47:04.470699 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.470668    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465e57d-0d08-4a48-9916-84e62d1dcfdd-plugins-config-volume\") pod \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") "
Aug 03 16:47:04.470825 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.470711    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7xkl\" (UniqueName: \"kubernetes.io/projected/4465e57d-0d08-4a48-9916-84e62d1dcfdd-kube-api-access-f7xkl\") pod \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\" (UID: \"4465e57d-0d08-4a48-9916-84e62d1dcfdd\") "
Aug 03 16:47:04.470825 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.470755    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88704101-31d9-4555-88b9-3449c539e2c9-openshift-service-ca\") pod \"88704101-31d9-4555-88b9-3449c539e2c9\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") "
Aug 03 16:47:04.470825 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.470779    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmd66\" (UniqueName: \"kubernetes.io/projected/88704101-31d9-4555-88b9-3449c539e2c9-kube-api-access-hmd66\") pod \"88704101-31d9-4555-88b9-3449c539e2c9\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") "
Aug 03 16:47:04.470825 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.470813    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls\") pod \"88704101-31d9-4555-88b9-3449c539e2c9\" (UID: \"88704101-31d9-4555-88b9-3449c539e2c9\") "
Aug 03 16:47:04.471198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.471171    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4465e57d-0d08-4a48-9916-84e62d1dcfdd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4465e57d-0d08-4a48-9916-84e62d1dcfdd" (UID: "4465e57d-0d08-4a48-9916-84e62d1dcfdd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:04.471293 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.471271    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88704101-31d9-4555-88b9-3449c539e2c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88704101-31d9-4555-88b9-3449c539e2c9" (UID: "88704101-31d9-4555-88b9-3449c539e2c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:04.471522 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.471499    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4465e57d-0d08-4a48-9916-84e62d1dcfdd-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.471661 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.471641    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88704101-31d9-4555-88b9-3449c539e2c9-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.473802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.473776    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4465e57d-0d08-4a48-9916-84e62d1dcfdd-kube-api-access-f7xkl" (OuterVolumeSpecName: "kube-api-access-f7xkl") pod "4465e57d-0d08-4a48-9916-84e62d1dcfdd" (UID: "4465e57d-0d08-4a48-9916-84e62d1dcfdd"). InnerVolumeSpecName "kube-api-access-f7xkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:04.473909 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.473823    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88704101-31d9-4555-88b9-3449c539e2c9" (UID: "88704101-31d9-4555-88b9-3449c539e2c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:47:04.474332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.474306    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88704101-31d9-4555-88b9-3449c539e2c9-kube-api-access-hmd66" (OuterVolumeSpecName: "kube-api-access-hmd66") pod "88704101-31d9-4555-88b9-3449c539e2c9" (UID: "88704101-31d9-4555-88b9-3449c539e2c9"). InnerVolumeSpecName "kube-api-access-hmd66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:04.571982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.571954    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-plugins-config-volume\") pod \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") "
Aug 03 16:47:04.572130 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.572076    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9vjv\" (UniqueName: \"kubernetes.io/projected/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-kube-api-access-r9vjv\") pod \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\" (UID: \"acae0d3c-8eef-4aa3-acca-4089f5fde0f0\") "
Aug 03 16:47:04.572297 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.572276    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "acae0d3c-8eef-4aa3-acca-4089f5fde0f0" (UID: "acae0d3c-8eef-4aa3-acca-4089f5fde0f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:04.572297 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.572290    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7xkl\" (UniqueName: \"kubernetes.io/projected/4465e57d-0d08-4a48-9916-84e62d1dcfdd-kube-api-access-f7xkl\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.572368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.572337    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmd66\" (UniqueName: \"kubernetes.io/projected/88704101-31d9-4555-88b9-3449c539e2c9-kube-api-access-hmd66\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.572368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.572357    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88704101-31d9-4555-88b9-3449c539e2c9-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.573934 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.573916    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-kube-api-access-r9vjv" (OuterVolumeSpecName: "kube-api-access-r9vjv") pod "acae0d3c-8eef-4aa3-acca-4089f5fde0f0" (UID: "acae0d3c-8eef-4aa3-acca-4089f5fde0f0"). InnerVolumeSpecName "kube-api-access-r9vjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:04.662056 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.661979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:04.673667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.673640    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9vjv\" (UniqueName: \"kubernetes.io/projected/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-kube-api-access-r9vjv\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.673667 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.673662    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acae0d3c-8eef-4aa3-acca-4089f5fde0f0-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:04.755722 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.755689    2560 generic.go:356] "Generic (PLEG): container finished" podID="88704101-31d9-4555-88b9-3449c539e2c9" containerID="1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a" exitCode=0
Aug 03 16:47:04.755897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.755745    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"
Aug 03 16:47:04.755897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.755763    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" event={"ID":"88704101-31d9-4555-88b9-3449c539e2c9","Type":"ContainerDied","Data":"1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a"}
Aug 03 16:47:04.755897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.755845    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj" event={"ID":"88704101-31d9-4555-88b9-3449c539e2c9","Type":"ContainerDied","Data":"349123cae1d80fd3ae1116bf92656a16d7d96cc0e9a6877626c2c487e64275da"}
Aug 03 16:47:04.755897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.755881    2560 scope.go:117] "RemoveContainer" containerID="1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a"
Aug 03 16:47:04.757243 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.757219    2560 generic.go:356] "Generic (PLEG): container finished" podID="4465e57d-0d08-4a48-9916-84e62d1dcfdd" containerID="558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7" exitCode=0
Aug 03 16:47:04.757368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.757291    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"
Aug 03 16:47:04.757368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.757296    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw" event={"ID":"4465e57d-0d08-4a48-9916-84e62d1dcfdd","Type":"ContainerDied","Data":"558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7"}
Aug 03 16:47:04.757368 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.757358    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw" event={"ID":"4465e57d-0d08-4a48-9916-84e62d1dcfdd","Type":"ContainerDied","Data":"09ba0e14c05f128f27761ea3381943fd306399a5614a9d114340ceda9a4d18d9"}
Aug 03 16:47:04.758694 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.758669    2560 generic.go:356] "Generic (PLEG): container finished" podID="acae0d3c-8eef-4aa3-acca-4089f5fde0f0" containerID="0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c" exitCode=0
Aug 03 16:47:04.758796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.758711    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"
Aug 03 16:47:04.758796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.758718    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr" event={"ID":"acae0d3c-8eef-4aa3-acca-4089f5fde0f0","Type":"ContainerDied","Data":"0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c"}
Aug 03 16:47:04.758796 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.758754    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr" event={"ID":"acae0d3c-8eef-4aa3-acca-4089f5fde0f0","Type":"ContainerDied","Data":"1e6a7c7c910609bb11044a7ae335ea2063257744fd55b40d3ec47e59d19d05c2"}
Aug 03 16:47:04.765565 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.765545    2560 scope.go:117] "RemoveContainer" containerID="1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a"
Aug 03 16:47:04.765849 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:04.765825    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a\": container with ID starting with 1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a not found: ID does not exist" containerID="1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a"
Aug 03 16:47:04.765943 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.765852    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a"} err="failed to get container status \"1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a\": rpc error: code = NotFound desc = could not find container \"1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a\": container with ID starting with 1560b530b4d0dc37fb5f98ce6b305088ad2096d0c7da55cb20cb57cef8d0b33a not found: ID does not exist"
Aug 03 16:47:04.765943 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.765870    2560 scope.go:117] "RemoveContainer" containerID="558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7"
Aug 03 16:47:04.774953 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.774934    2560 scope.go:117] "RemoveContainer" containerID="558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7"
Aug 03 16:47:04.775227 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:04.775210    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7\": container with ID starting with 558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7 not found: ID does not exist" containerID="558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7"
Aug 03 16:47:04.775278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.775234    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7"} err="failed to get container status \"558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7\": rpc error: code = NotFound desc = could not find container \"558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7\": container with ID starting with 558049982b57dff489f0c10615c5362dd45df18b33620e0d18beb317e1d0aea7 not found: ID does not exist"
Aug 03 16:47:04.775278 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.775250    2560 scope.go:117] "RemoveContainer" containerID="0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c"
Aug 03 16:47:04.782785 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.782771    2560 scope.go:117] "RemoveContainer" containerID="0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c"
Aug 03 16:47:04.783014 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:04.782996    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c\": container with ID starting with 0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c not found: ID does not exist" containerID="0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c"
Aug 03 16:47:04.783065 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.783020    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c"} err="failed to get container status \"0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c\": rpc error: code = NotFound desc = could not find container \"0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c\": container with ID starting with 0232c24c496fb14119c6c900e24b1fc442134174423e1a8609219ec0bc63001c not found: ID does not exist"
Aug 03 16:47:04.787385 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.787361    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"]
Aug 03 16:47:04.791401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.791382    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7ca182-7b5bb5cf76-48mqj"]
Aug 03 16:47:04.802649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.802628    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"]
Aug 03 16:47:04.807793 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.807774    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ca182-55b4546964-qxvkr"]
Aug 03 16:47:04.820449 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.820430    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"]
Aug 03 16:47:04.825185 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.825168    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ca182-766b96f5fd-fl6rw"]
Aug 03 16:47:04.986797 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.986717    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15120425-d678-472e-b08c-d9f86038c666" path="/var/lib/kubelet/pods/15120425-d678-472e-b08c-d9f86038c666/volumes"
Aug 03 16:47:04.987042 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.987028    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38679053-2e5d-41a2-ba8b-66b41d602d29" path="/var/lib/kubelet/pods/38679053-2e5d-41a2-ba8b-66b41d602d29/volumes"
Aug 03 16:47:04.987339 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.987325    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4465e57d-0d08-4a48-9916-84e62d1dcfdd" path="/var/lib/kubelet/pods/4465e57d-0d08-4a48-9916-84e62d1dcfdd/volumes"
Aug 03 16:47:04.987724 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.987710    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88704101-31d9-4555-88b9-3449c539e2c9" path="/var/lib/kubelet/pods/88704101-31d9-4555-88b9-3449c539e2c9/volumes"
Aug 03 16:47:04.988270 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:04.988254    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acae0d3c-8eef-4aa3-acca-4089f5fde0f0" path="/var/lib/kubelet/pods/acae0d3c-8eef-4aa3-acca-4089f5fde0f0/volumes"
Aug 03 16:47:10.854023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:10.853985    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:10.911545 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:10.911496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:11.143901 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.143826    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:11.373080 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.373051    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:11.483441 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483362    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"]
Aug 03 16:47:11.483801 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483786    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4465e57d-0d08-4a48-9916-84e62d1dcfdd" containerName="payload-processing"
Aug 03 16:47:11.483845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483803    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4465e57d-0d08-4a48-9916-84e62d1dcfdd" containerName="payload-processing"
Aug 03 16:47:11.483845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483818    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88704101-31d9-4555-88b9-3449c539e2c9" containerName="maas-api"
Aug 03 16:47:11.483845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483824    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="88704101-31d9-4555-88b9-3449c539e2c9" containerName="maas-api"
Aug 03 16:47:11.483845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483834    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15120425-d678-472e-b08c-d9f86038c666" containerName="revoke-keys"
Aug 03 16:47:11.483845 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483840    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15120425-d678-472e-b08c-d9f86038c666" containerName="revoke-keys"
Aug 03 16:47:11.483996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483851    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acae0d3c-8eef-4aa3-acca-4089f5fde0f0" containerName="payload-pre-processing"
Aug 03 16:47:11.483996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483857    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="acae0d3c-8eef-4aa3-acca-4089f5fde0f0" containerName="payload-pre-processing"
Aug 03 16:47:11.483996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483919    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="15120425-d678-472e-b08c-d9f86038c666" containerName="revoke-keys"
Aug 03 16:47:11.483996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483927    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="acae0d3c-8eef-4aa3-acca-4089f5fde0f0" containerName="payload-pre-processing"
Aug 03 16:47:11.483996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483934    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4465e57d-0d08-4a48-9916-84e62d1dcfdd" containerName="payload-processing"
Aug 03 16:47:11.483996 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.483943    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="88704101-31d9-4555-88b9-3449c539e2c9" containerName="maas-api"
Aug 03 16:47:11.487137 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.487121    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:11.489577 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.489553    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:47:11.494370 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.494343    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"]
Aug 03 16:47:11.529339 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.529310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/186cb682-53a3-454d-a45f-504fa4cf7e12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk\" (UID: \"186cb682-53a3-454d-a45f-504fa4cf7e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:11.629826 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.629798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/186cb682-53a3-454d-a45f-504fa4cf7e12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk\" (UID: \"186cb682-53a3-454d-a45f-504fa4cf7e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:11.630538 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.630501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/186cb682-53a3-454d-a45f-504fa4cf7e12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk\" (UID: \"186cb682-53a3-454d-a45f-504fa4cf7e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:11.799820 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.799796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:11.945066 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:11.945040    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"]
Aug 03 16:47:11.947169 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:47:11.947136    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186cb682_53a3_454d_a45f_504fa4cf7e12.slice/crio-28991772a58517c4012366c6fd90ab4d11582d4547e84358a498109b6af5bdcc WatchSource:0}: Error finding container 28991772a58517c4012366c6fd90ab4d11582d4547e84358a498109b6af5bdcc: Status 404 returned error can't find the container with id 28991772a58517c4012366c6fd90ab4d11582d4547e84358a498109b6af5bdcc
Aug 03 16:47:12.804883 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:12.804845    2560 generic.go:356] "Generic (PLEG): container finished" podID="186cb682-53a3-454d-a45f-504fa4cf7e12" containerID="8be9b87f207e9c8193c2d5d76db23c2b777dc7b1e1bf8f8b41d865f7f20e9522" exitCode=0
Aug 03 16:47:12.805059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:12.804891    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk" event={"ID":"186cb682-53a3-454d-a45f-504fa4cf7e12","Type":"ContainerDied","Data":"8be9b87f207e9c8193c2d5d76db23c2b777dc7b1e1bf8f8b41d865f7f20e9522"}
Aug 03 16:47:12.805059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:12.804935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk" event={"ID":"186cb682-53a3-454d-a45f-504fa4cf7e12","Type":"ContainerStarted","Data":"28991772a58517c4012366c6fd90ab4d11582d4547e84358a498109b6af5bdcc"}
Aug 03 16:47:13.948406 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:13.948381    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:14.051249 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:14.051217    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/186cb682-53a3-454d-a45f-504fa4cf7e12-maas-api-service-ca\") pod \"186cb682-53a3-454d-a45f-504fa4cf7e12\" (UID: \"186cb682-53a3-454d-a45f-504fa4cf7e12\") "
Aug 03 16:47:14.051608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:14.051587    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186cb682-53a3-454d-a45f-504fa4cf7e12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "186cb682-53a3-454d-a45f-504fa4cf7e12" (UID: "186cb682-53a3-454d-a45f-504fa4cf7e12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:14.152706 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:14.152628    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/186cb682-53a3-454d-a45f-504fa4cf7e12-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:14.815873 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:14.815840    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"
Aug 03 16:47:14.816053 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:14.815881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk" event={"ID":"186cb682-53a3-454d-a45f-504fa4cf7e12","Type":"ContainerDied","Data":"28991772a58517c4012366c6fd90ab4d11582d4547e84358a498109b6af5bdcc"}
Aug 03 16:47:14.816053 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:14.815927    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28991772a58517c4012366c6fd90ab4d11582d4547e84358a498109b6af5bdcc"
Aug 03 16:47:21.889573 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.889330    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"]
Aug 03 16:47:21.890078 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.889597    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq" podUID="f17876e5-e23b-4d92-bdde-01a5d259a667" containerName="maas-api" containerID="cri-o://d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f" gracePeriod=30
Aug 03 16:47:21.913124 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.913097    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:21.949470 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.949433    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"]
Aug 03 16:47:21.949792 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.949740    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg" podUID="75f580d1-910b-491a-b8c1-237ffb1ad047" containerName="payload-processing" containerID="cri-o://90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de" gracePeriod=30
Aug 03 16:47:21.957398 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.957366    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"]
Aug 03 16:47:21.957638 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:21.957615    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr" podUID="d4284b30-fb0a-4f4f-b339-53cb5b73635d" containerName="payload-pre-processing" containerID="cri-o://cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123" gracePeriod=30
Aug 03 16:47:22.010486 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.010454    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:22.106144 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.106096    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:22.118889 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.118828    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"]
Aug 03 16:47:22.122102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.122076    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d7cee2-gggkk"]
Aug 03 16:47:22.218141 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.218116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:22.243046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.243020    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:47:22.278499 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.278470    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:47:22.281728 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.281708    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:47:22.324154 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324128    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f17876e5-e23b-4d92-bdde-01a5d259a667-maas-api-tls\") pod \"f17876e5-e23b-4d92-bdde-01a5d259a667\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") "
Aug 03 16:47:22.324315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324165    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4284b30-fb0a-4f4f-b339-53cb5b73635d-plugins-config-volume\") pod \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") "
Aug 03 16:47:22.324315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324194    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg8nz\" (UniqueName: \"kubernetes.io/projected/f17876e5-e23b-4d92-bdde-01a5d259a667-kube-api-access-zg8nz\") pod \"f17876e5-e23b-4d92-bdde-01a5d259a667\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") "
Aug 03 16:47:22.324315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324214    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75f580d1-910b-491a-b8c1-237ffb1ad047-plugins-config-volume\") pod \"75f580d1-910b-491a-b8c1-237ffb1ad047\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") "
Aug 03 16:47:22.324315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324276    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17876e5-e23b-4d92-bdde-01a5d259a667-openshift-service-ca\") pod \"f17876e5-e23b-4d92-bdde-01a5d259a667\" (UID: \"f17876e5-e23b-4d92-bdde-01a5d259a667\") "
Aug 03 16:47:22.324507 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324330    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pb662\" (UniqueName: \"kubernetes.io/projected/d4284b30-fb0a-4f4f-b339-53cb5b73635d-kube-api-access-pb662\") pod \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\" (UID: \"d4284b30-fb0a-4f4f-b339-53cb5b73635d\") "
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324702    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17876e5-e23b-4d92-bdde-01a5d259a667-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f17876e5-e23b-4d92-bdde-01a5d259a667" (UID: "f17876e5-e23b-4d92-bdde-01a5d259a667"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324708    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f580d1-910b-491a-b8c1-237ffb1ad047-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75f580d1-910b-491a-b8c1-237ffb1ad047" (UID: "75f580d1-910b-491a-b8c1-237ffb1ad047"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324822    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4284b30-fb0a-4f4f-b339-53cb5b73635d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4284b30-fb0a-4f4f-b339-53cb5b73635d" (UID: "d4284b30-fb0a-4f4f-b339-53cb5b73635d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.324899    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd4zt\" (UniqueName: \"kubernetes.io/projected/75f580d1-910b-491a-b8c1-237ffb1ad047-kube-api-access-cd4zt\") pod \"75f580d1-910b-491a-b8c1-237ffb1ad047\" (UID: \"75f580d1-910b-491a-b8c1-237ffb1ad047\") "
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.325263    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17876e5-e23b-4d92-bdde-01a5d259a667-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.325298    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4284b30-fb0a-4f4f-b339-53cb5b73635d-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.325359 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.325314    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75f580d1-910b-491a-b8c1-237ffb1ad047-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.326510 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.326481    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17876e5-e23b-4d92-bdde-01a5d259a667-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f17876e5-e23b-4d92-bdde-01a5d259a667" (UID: "f17876e5-e23b-4d92-bdde-01a5d259a667"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:47:22.326762 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.326739    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4284b30-fb0a-4f4f-b339-53cb5b73635d-kube-api-access-pb662" (OuterVolumeSpecName: "kube-api-access-pb662") pod "d4284b30-fb0a-4f4f-b339-53cb5b73635d" (UID: "d4284b30-fb0a-4f4f-b339-53cb5b73635d"). InnerVolumeSpecName "kube-api-access-pb662". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:22.327072 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.327054    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17876e5-e23b-4d92-bdde-01a5d259a667-kube-api-access-zg8nz" (OuterVolumeSpecName: "kube-api-access-zg8nz") pod "f17876e5-e23b-4d92-bdde-01a5d259a667" (UID: "f17876e5-e23b-4d92-bdde-01a5d259a667"). InnerVolumeSpecName "kube-api-access-zg8nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:22.327287 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.327262    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75f580d1-910b-491a-b8c1-237ffb1ad047-kube-api-access-cd4zt" (OuterVolumeSpecName: "kube-api-access-cd4zt") pod "75f580d1-910b-491a-b8c1-237ffb1ad047" (UID: "75f580d1-910b-491a-b8c1-237ffb1ad047"). InnerVolumeSpecName "kube-api-access-cd4zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:47:22.426396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.426295    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f17876e5-e23b-4d92-bdde-01a5d259a667-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.426396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.426330    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg8nz\" (UniqueName: \"kubernetes.io/projected/f17876e5-e23b-4d92-bdde-01a5d259a667-kube-api-access-zg8nz\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.426396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.426344    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pb662\" (UniqueName: \"kubernetes.io/projected/d4284b30-fb0a-4f4f-b339-53cb5b73635d-kube-api-access-pb662\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.426396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.426358    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd4zt\" (UniqueName: \"kubernetes.io/projected/75f580d1-910b-491a-b8c1-237ffb1ad047-kube-api-access-cd4zt\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:47:22.495227 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.495190    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:22.858542 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.858495    2560 generic.go:356] "Generic (PLEG): container finished" podID="d4284b30-fb0a-4f4f-b339-53cb5b73635d" containerID="cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123" exitCode=0
Aug 03 16:47:22.858746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.858568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"
Aug 03 16:47:22.858746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.858584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr" event={"ID":"d4284b30-fb0a-4f4f-b339-53cb5b73635d","Type":"ContainerDied","Data":"cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123"}
Aug 03 16:47:22.858746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.858638    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr" event={"ID":"d4284b30-fb0a-4f4f-b339-53cb5b73635d","Type":"ContainerDied","Data":"62d17d5c3518682e27c60743f42c1552426536580f63f968643612abf3f9319d"}
Aug 03 16:47:22.858746 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.858664    2560 scope.go:117] "RemoveContainer" containerID="cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123"
Aug 03 16:47:22.859896 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.859875    2560 generic.go:356] "Generic (PLEG): container finished" podID="75f580d1-910b-491a-b8c1-237ffb1ad047" containerID="90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de" exitCode=0
Aug 03 16:47:22.859990 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.859919    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"
Aug 03 16:47:22.859990 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.859963    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg" event={"ID":"75f580d1-910b-491a-b8c1-237ffb1ad047","Type":"ContainerDied","Data":"90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de"}
Aug 03 16:47:22.860099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.860013    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg" event={"ID":"75f580d1-910b-491a-b8c1-237ffb1ad047","Type":"ContainerDied","Data":"5e738d7b8322a6e21fafa5fec4e289b7ca5d2849326d9bfed0d1e9db06ff64c9"}
Aug 03 16:47:22.861568 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.861545    2560 generic.go:356] "Generic (PLEG): container finished" podID="f17876e5-e23b-4d92-bdde-01a5d259a667" containerID="d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f" exitCode=0
Aug 03 16:47:22.861683 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.861604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq" event={"ID":"f17876e5-e23b-4d92-bdde-01a5d259a667","Type":"ContainerDied","Data":"d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f"}
Aug 03 16:47:22.861683 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.861642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq" event={"ID":"f17876e5-e23b-4d92-bdde-01a5d259a667","Type":"ContainerDied","Data":"e5310e31a5dd33542082634c7e0fe99c1f2c836852f8e313dab5d793e7e1fc5c"}
Aug 03 16:47:22.861683 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.861644    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"
Aug 03 16:47:22.868725 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.868706    2560 scope.go:117] "RemoveContainer" containerID="cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123"
Aug 03 16:47:22.869020 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:22.868996    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123\": container with ID starting with cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123 not found: ID does not exist" containerID="cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123"
Aug 03 16:47:22.869103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.869030    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123"} err="failed to get container status \"cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123\": rpc error: code = NotFound desc = could not find container \"cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123\": container with ID starting with cf92cb69d3ed6b5ea703ec30ace64fdc8d8dbf1bcc246a59c45a50b1dd017123 not found: ID does not exist"
Aug 03 16:47:22.869103 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.869056    2560 scope.go:117] "RemoveContainer" containerID="90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de"
Aug 03 16:47:22.879194 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.879176    2560 scope.go:117] "RemoveContainer" containerID="90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de"
Aug 03 16:47:22.879459 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:22.879440    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de\": container with ID starting with 90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de not found: ID does not exist" containerID="90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de"
Aug 03 16:47:22.879627 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.879469    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de"} err="failed to get container status \"90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de\": rpc error: code = NotFound desc = could not find container \"90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de\": container with ID starting with 90df8ccb77635448d0f1cccea7b7789b6dbf0b4d66951fe94db6e99695c059de not found: ID does not exist"
Aug 03 16:47:22.879627 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.879496    2560 scope.go:117] "RemoveContainer" containerID="d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f"
Aug 03 16:47:22.890194 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.888863    2560 scope.go:117] "RemoveContainer" containerID="d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f"
Aug 03 16:47:22.890194 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:22.889253    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f\": container with ID starting with d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f not found: ID does not exist" containerID="d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f"
Aug 03 16:47:22.890194 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.889290    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f"} err="failed to get container status \"d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f\": rpc error: code = NotFound desc = could not find container \"d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f\": container with ID starting with d164752ee9c85f6e0112917f49baf40fe7fc284cafe0208b5acce942634e296f not found: ID does not exist"
Aug 03 16:47:22.891591 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.890920    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"]
Aug 03 16:47:22.896203 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.896178    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d7cee2-77c7977944-7szjr"]
Aug 03 16:47:22.905207 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.905133    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"]
Aug 03 16:47:22.908403 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.908369    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d7cee2-65467cf884-knptg"]
Aug 03 16:47:22.918654 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.918626    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"]
Aug 03 16:47:22.924739 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.924711    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7cee2-7cf6dcbb87-qcpgq"]
Aug 03 16:47:22.987596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.987561    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186cb682-53a3-454d-a45f-504fa4cf7e12" path="/var/lib/kubelet/pods/186cb682-53a3-454d-a45f-504fa4cf7e12/volumes"
Aug 03 16:47:22.987971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.987958    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75f580d1-910b-491a-b8c1-237ffb1ad047" path="/var/lib/kubelet/pods/75f580d1-910b-491a-b8c1-237ffb1ad047/volumes"
Aug 03 16:47:22.988303 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.988291    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4284b30-fb0a-4f4f-b339-53cb5b73635d" path="/var/lib/kubelet/pods/d4284b30-fb0a-4f4f-b339-53cb5b73635d/volumes"
Aug 03 16:47:22.988677 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:22.988665    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f17876e5-e23b-4d92-bdde-01a5d259a667" path="/var/lib/kubelet/pods/f17876e5-e23b-4d92-bdde-01a5d259a667/volumes"
Aug 03 16:47:28.828963 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:28.828933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:28.930322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:28.930282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:29.118816 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:29.118727    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:29.338612 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:29.338576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:29.680085 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:29.680049    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:30.002851 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:30.002766    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:30.163351 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:30.163314    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:33.080319 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:33.080285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:33.187477 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:33.187437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:34.168669 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.168630    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"]
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169027    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4284b30-fb0a-4f4f-b339-53cb5b73635d" containerName="payload-pre-processing"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169041    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4284b30-fb0a-4f4f-b339-53cb5b73635d" containerName="payload-pre-processing"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169050    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186cb682-53a3-454d-a45f-504fa4cf7e12" containerName="revoke-keys"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169056    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="186cb682-53a3-454d-a45f-504fa4cf7e12" containerName="revoke-keys"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169065    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75f580d1-910b-491a-b8c1-237ffb1ad047" containerName="payload-processing"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169072    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="75f580d1-910b-491a-b8c1-237ffb1ad047" containerName="payload-processing"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169086    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17876e5-e23b-4d92-bdde-01a5d259a667" containerName="maas-api"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169091    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17876e5-e23b-4d92-bdde-01a5d259a667" containerName="maas-api"
Aug 03 16:47:34.169157 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169164    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="186cb682-53a3-454d-a45f-504fa4cf7e12" containerName="revoke-keys"
Aug 03 16:47:34.170036 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169171    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17876e5-e23b-4d92-bdde-01a5d259a667" containerName="maas-api"
Aug 03 16:47:34.170036 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169179    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="75f580d1-910b-491a-b8c1-237ffb1ad047" containerName="payload-processing"
Aug 03 16:47:34.170036 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.169184    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4284b30-fb0a-4f4f-b339-53cb5b73635d" containerName="payload-pre-processing"
Aug 03 16:47:34.173849 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.173829    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.176557 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.176521    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-deec1a-serving-cert\""
Aug 03 16:47:34.184521 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.184427    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"]
Aug 03 16:47:34.198769 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.198734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:34.334306 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.334268    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.334498 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.334330    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvfz\" (UniqueName: \"kubernetes.io/projected/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-kube-api-access-gjvfz\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.334498 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.334389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-openshift-service-ca\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.435464 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.435370    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.435464 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.435428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvfz\" (UniqueName: \"kubernetes.io/projected/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-kube-api-access-gjvfz\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.435722 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.435470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-openshift-service-ca\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.435722 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:34.435552    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-serving-cert: secret "maas-api-e2e-api-a-deec1a-serving-cert" not found
Aug 03 16:47:34.435722 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:47:34.435639    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls podName:ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce nodeName:}" failed. No retries permitted until 2026-08-03 16:47:34.935619826 +0000 UTC m=+2614.484906120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls") pod "maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" (UID: "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce") : secret "maas-api-e2e-api-a-deec1a-serving-cert" not found
Aug 03 16:47:34.436198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.436180    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-openshift-service-ca\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.444908 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.444876    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvfz\" (UniqueName: \"kubernetes.io/projected/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-kube-api-access-gjvfz\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.699334 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.699242    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:34.794403 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.794361    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:34.808797 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.808761    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"]
Aug 03 16:47:34.812497 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.812481    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:34.815154 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.815129    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-deec1a\""
Aug 03 16:47:34.815654 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.815638    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-deec1a-dockercfg-4mnpg\""
Aug 03 16:47:34.816386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.816363    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"]
Aug 03 16:47:34.819777 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.819755    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:34.824500 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.824474    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"]
Aug 03 16:47:34.829727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.829703    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"]
Aug 03 16:47:34.894905 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.894869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:34.941385 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.941343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz7fr\" (UniqueName: \"kubernetes.io/projected/148acf54-c742-4253-b7ae-52aec760b22f-kube-api-access-jz7fr\") pod \"payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") " pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:34.941615 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.941404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:34.941615 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.941444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:34.941615 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.941493    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/148acf54-c742-4253-b7ae-52aec760b22f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") " pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:34.941802 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.941623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfjdg\" (UniqueName: \"kubernetes.io/projected/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-kube-api-access-gfjdg\") pod \"payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:34.943851 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:34.943830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls\") pod \"maas-api-e2e-api-a-deec1a-989f666d6-xmvt5\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:35.043001 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.042966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:35.043180 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.043023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/148acf54-c742-4253-b7ae-52aec760b22f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") " pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:35.043180 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.043073    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfjdg\" (UniqueName: \"kubernetes.io/projected/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-kube-api-access-gfjdg\") pod \"payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:35.043180 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.043155    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz7fr\" (UniqueName: \"kubernetes.io/projected/148acf54-c742-4253-b7ae-52aec760b22f-kube-api-access-jz7fr\") pod \"payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") " pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:35.043784 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.043767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:35.043830 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.043802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/148acf54-c742-4253-b7ae-52aec760b22f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") " pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:35.052139 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.052117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfjdg\" (UniqueName: \"kubernetes.io/projected/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-kube-api-access-gfjdg\") pod \"payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:35.052294 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.052165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz7fr\" (UniqueName: \"kubernetes.io/projected/148acf54-c742-4253-b7ae-52aec760b22f-kube-api-access-jz7fr\") pod \"payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") " pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:35.085874 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.085845    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:35.126311 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.126285    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:35.135812 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.135790    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:35.248328 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.248299    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"]
Aug 03 16:47:35.249864 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:47:35.249797    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0fc6b2_ba5c_4379_bd3d_06e55dec49ce.slice/crio-c11dc27533b70dd2c7de27ed2c0dbf2155a7661884d02aebe51637a68f1f9e22 WatchSource:0}: Error finding container c11dc27533b70dd2c7de27ed2c0dbf2155a7661884d02aebe51637a68f1f9e22: Status 404 returned error can't find the container with id c11dc27533b70dd2c7de27ed2c0dbf2155a7661884d02aebe51637a68f1f9e22
Aug 03 16:47:35.251482 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.251459    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:47:35.299401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.299373    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"]
Aug 03 16:47:35.300767 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:47:35.300726    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9bd61a5_7c2c_4663_bcc1_86dbdc306edf.slice/crio-8c9d6d643f07c5bf106e174590632f6b6d3c756329db67216ff25bec91722a26 WatchSource:0}: Error finding container 8c9d6d643f07c5bf106e174590632f6b6d3c756329db67216ff25bec91722a26: Status 404 returned error can't find the container with id 8c9d6d643f07c5bf106e174590632f6b6d3c756329db67216ff25bec91722a26
Aug 03 16:47:35.529106 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.529077    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"]
Aug 03 16:47:35.530966 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:47:35.530938    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148acf54_c742_4253_b7ae_52aec760b22f.slice/crio-9d7e8b048623b243551176bc480b11b88c46fa9c7e3874ef562462c66229e124 WatchSource:0}: Error finding container 9d7e8b048623b243551176bc480b11b88c46fa9c7e3874ef562462c66229e124: Status 404 returned error can't find the container with id 9d7e8b048623b243551176bc480b11b88c46fa9c7e3874ef562462c66229e124
Aug 03 16:47:35.939799 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.939688    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9" event={"ID":"148acf54-c742-4253-b7ae-52aec760b22f","Type":"ContainerStarted","Data":"eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707"}
Aug 03 16:47:35.939799 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.939754    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9" event={"ID":"148acf54-c742-4253-b7ae-52aec760b22f","Type":"ContainerStarted","Data":"9d7e8b048623b243551176bc480b11b88c46fa9c7e3874ef562462c66229e124"}
Aug 03 16:47:35.940009 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.939887    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:35.941437 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.941406    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9" event={"ID":"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf","Type":"ContainerStarted","Data":"2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7"}
Aug 03 16:47:35.941594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.941444    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9" event={"ID":"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf","Type":"ContainerStarted","Data":"8c9d6d643f07c5bf106e174590632f6b6d3c756329db67216ff25bec91722a26"}
Aug 03 16:47:35.941594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.941503    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:35.943203 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.943175    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" event={"ID":"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce","Type":"ContainerStarted","Data":"aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df"}
Aug 03 16:47:35.943315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.943215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" event={"ID":"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce","Type":"ContainerStarted","Data":"c11dc27533b70dd2c7de27ed2c0dbf2155a7661884d02aebe51637a68f1f9e22"}
Aug 03 16:47:35.943315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.943234    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:35.956575 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.956502    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9" podStartSLOduration=1.956486372 podStartE2EDuration="1.956486372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:47:35.955390726 +0000 UTC m=+2615.504677044" watchObservedRunningTime="2026-08-03 16:47:35.956486372 +0000 UTC m=+2615.505772687"
Aug 03 16:47:35.973948 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.973896    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" podStartSLOduration=1.477937394 podStartE2EDuration="1.973876457s" totalImagesPullingTime="495.939063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:47:35.972053115 +0000 UTC m=+2615.521339443" watchObservedRunningTime="2026-08-03 16:47:35.973876457 +0000 UTC m=+2615.523162806"
Aug 03 16:47:35.987736 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:35.987676    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9" podStartSLOduration=1.9876598749999999 podStartE2EDuration="1.987659875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:47:35.986865378 +0000 UTC m=+2615.536151711" watchObservedRunningTime="2026-08-03 16:47:35.987659875 +0000 UTC m=+2615.536946188"
Aug 03 16:47:41.956169 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:41.956134    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:47:46.952550 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:46.952507    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:47:46.953131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:46.952679    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:47:58.093374 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:58.093335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:58.587181 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:58.587146    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:58.909423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:58.909341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:47:59.544024 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:47:59.543997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:02.276068 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:02.276037    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:02.368032 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:02.368003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:03.473486 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.473450    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"]
Aug 03 16:48:03.479148 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.479122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.482663 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.482642    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9ffb05-serving-cert\""
Aug 03 16:48:03.488966 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.488866    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"]
Aug 03 16:48:03.529232 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.529201    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:03.588992 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.588958    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8998e4f7-cc82-492b-b93c-8183fdbaab52-openshift-service-ca\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.589160 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.589014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxjf\" (UniqueName: \"kubernetes.io/projected/8998e4f7-cc82-492b-b93c-8183fdbaab52-kube-api-access-4pxjf\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.589160 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.589046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.690554 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.690493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8998e4f7-cc82-492b-b93c-8183fdbaab52-openshift-service-ca\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.690730 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.690582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxjf\" (UniqueName: \"kubernetes.io/projected/8998e4f7-cc82-492b-b93c-8183fdbaab52-kube-api-access-4pxjf\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.690730 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.690603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.690730 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:48:03.690717    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-serving-cert: secret "maas-api-e2e-api-b-9ffb05-serving-cert" not found
Aug 03 16:48:03.690928 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:48:03.690785    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls podName:8998e4f7-cc82-492b-b93c-8183fdbaab52 nodeName:}" failed. No retries permitted until 2026-08-03 16:48:04.190764195 +0000 UTC m=+2643.740050502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls") pod "maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" (UID: "8998e4f7-cc82-492b-b93c-8183fdbaab52") : secret "maas-api-e2e-api-b-9ffb05-serving-cert" not found
Aug 03 16:48:03.691140 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.691120    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8998e4f7-cc82-492b-b93c-8183fdbaab52-openshift-service-ca\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:03.700236 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:03.700212    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxjf\" (UniqueName: \"kubernetes.io/projected/8998e4f7-cc82-492b-b93c-8183fdbaab52-kube-api-access-4pxjf\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:04.100133 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.100102    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"]
Aug 03 16:48:04.103728 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.103712    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.106309 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.106285    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9ffb05\""
Aug 03 16:48:04.106420 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.106346    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9ffb05-dockercfg-4kj7g\""
Aug 03 16:48:04.115556 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.115511    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"]
Aug 03 16:48:04.156663 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.156631    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"]
Aug 03 16:48:04.160063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.160047    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.170114 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.170087    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"]
Aug 03 16:48:04.194999 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.194967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:04.195146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.195023    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.195146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.195088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-675d6\" (UniqueName: \"kubernetes.io/projected/5fc6d07f-2765-48db-a0a4-dad080f08161-kube-api-access-675d6\") pod \"payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") " pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.195146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.195119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc6d07f-2765-48db-a0a4-dad080f08161-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") " pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.195329 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.195189    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmpf2\" (UniqueName: \"kubernetes.io/projected/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-kube-api-access-qmpf2\") pod \"payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.198177 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.198154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls\") pod \"maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:04.296508 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.296471    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.296726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.296537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-675d6\" (UniqueName: \"kubernetes.io/projected/5fc6d07f-2765-48db-a0a4-dad080f08161-kube-api-access-675d6\") pod \"payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") " pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.296726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.296570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc6d07f-2765-48db-a0a4-dad080f08161-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") " pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.296726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.296623    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmpf2\" (UniqueName: \"kubernetes.io/projected/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-kube-api-access-qmpf2\") pod \"payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.297309 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.297287    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.297461 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.297439    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc6d07f-2765-48db-a0a4-dad080f08161-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") " pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.305875 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.305844    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmpf2\" (UniqueName: \"kubernetes.io/projected/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-kube-api-access-qmpf2\") pod \"payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.305977 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.305877    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-675d6\" (UniqueName: \"kubernetes.io/projected/5fc6d07f-2765-48db-a0a4-dad080f08161-kube-api-access-675d6\") pod \"payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") " pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.394629 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.394553    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:04.414816 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.414788    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:04.471799 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.471768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:04.553155 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.552614    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"]
Aug 03 16:48:04.639576 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.639547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"]
Aug 03 16:48:04.641542 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:48:04.641505    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc6d07f_2765_48db_a0a4_dad080f08161.slice/crio-f7aac694d9d898008ab81da9cc7eca3dc46a52f8ba90e70eb1067a75c79525e1 WatchSource:0}: Error finding container f7aac694d9d898008ab81da9cc7eca3dc46a52f8ba90e70eb1067a75c79525e1: Status 404 returned error can't find the container with id f7aac694d9d898008ab81da9cc7eca3dc46a52f8ba90e70eb1067a75c79525e1
Aug 03 16:48:04.789800 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:04.789775    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"]
Aug 03 16:48:04.791752 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:48:04.791725    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf724a542_a5f2_4d17_b2c5_5c6d34a75ff0.slice/crio-fc301a5e07cb7e72b2d6b2eb2b0b584b7fedc15ecab276924ccc284735ab18e1 WatchSource:0}: Error finding container fc301a5e07cb7e72b2d6b2eb2b0b584b7fedc15ecab276924ccc284735ab18e1: Status 404 returned error can't find the container with id fc301a5e07cb7e72b2d6b2eb2b0b584b7fedc15ecab276924ccc284735ab18e1
Aug 03 16:48:05.040710 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.040681    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:05.106881 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.106850    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8" event={"ID":"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0","Type":"ContainerStarted","Data":"7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23"}
Aug 03 16:48:05.107062 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.106913    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8" event={"ID":"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0","Type":"ContainerStarted","Data":"fc301a5e07cb7e72b2d6b2eb2b0b584b7fedc15ecab276924ccc284735ab18e1"}
Aug 03 16:48:05.107131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.107072    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:05.109360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.109244    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" event={"ID":"8998e4f7-cc82-492b-b93c-8183fdbaab52","Type":"ContainerStarted","Data":"398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691"}
Aug 03 16:48:05.109360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.109304    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:05.109360 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.109330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" event={"ID":"8998e4f7-cc82-492b-b93c-8183fdbaab52","Type":"ContainerStarted","Data":"8ebe3886fe187cd23d745d97f8d4ff35cbfddd06c477907152dd20eccfdc6685"}
Aug 03 16:48:05.111506 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.111483    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s" event={"ID":"5fc6d07f-2765-48db-a0a4-dad080f08161","Type":"ContainerStarted","Data":"a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b"}
Aug 03 16:48:05.111640 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.111561    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s" event={"ID":"5fc6d07f-2765-48db-a0a4-dad080f08161","Type":"ContainerStarted","Data":"f7aac694d9d898008ab81da9cc7eca3dc46a52f8ba90e70eb1067a75c79525e1"}
Aug 03 16:48:05.111640 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.111593    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:05.137190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.137149    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8" podStartSLOduration=1.137136996 podStartE2EDuration="1.137136996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:48:05.134264985 +0000 UTC m=+2644.683551302" watchObservedRunningTime="2026-08-03 16:48:05.137136996 +0000 UTC m=+2644.686423312"
Aug 03 16:48:05.155443 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.155402    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" podStartSLOduration=1.824559622 podStartE2EDuration="2.155387274s" totalImagesPullingTime="330.827652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:48:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:48:05.155095587 +0000 UTC m=+2644.704381904" watchObservedRunningTime="2026-08-03 16:48:05.155387274 +0000 UTC m=+2644.704673597"
Aug 03 16:48:05.173017 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:05.172942    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s" podStartSLOduration=1.172929912 podStartE2EDuration="1.172929912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:48:05.171724859 +0000 UTC m=+2644.721011176" watchObservedRunningTime="2026-08-03 16:48:05.172929912 +0000 UTC m=+2644.722216258"
Aug 03 16:48:08.187384 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:08.187345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:08.289236 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:08.289195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:11.123045 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:11.123013    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:48:16.119099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:16.119060    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:48:16.119476 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:16.119129    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:48:27.292117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:27.292083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:41.430168 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.430128    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"]
Aug 03 16:48:41.430677 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.430411    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" podUID="ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" containerName="maas-api" containerID="cri-o://aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df" gracePeriod=30
Aug 03 16:48:41.477025 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.476990    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:41.506046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.506014    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"]
Aug 03 16:48:41.506247 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.506225    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9" podUID="148acf54-c742-4253-b7ae-52aec760b22f" containerName="payload-processing" containerID="cri-o://eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707" gracePeriod=30
Aug 03 16:48:41.511984 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.511956    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"]
Aug 03 16:48:41.512267 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.512223    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9" podUID="d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" containerName="payload-pre-processing" containerID="cri-o://2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7" gracePeriod=30
Aug 03 16:48:41.601519 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.601490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:41.722322 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.722283    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:41.800586 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.800559    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:48:41.812654 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.812632    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:48:41.847804 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.847776    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:48:41.951682 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951647    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls\") pod \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") "
Aug 03 16:48:41.951897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951725    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-plugins-config-volume\") pod \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") "
Aug 03 16:48:41.951897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951748    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfjdg\" (UniqueName: \"kubernetes.io/projected/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-kube-api-access-gfjdg\") pod \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\" (UID: \"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf\") "
Aug 03 16:48:41.951897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951768    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz7fr\" (UniqueName: \"kubernetes.io/projected/148acf54-c742-4253-b7ae-52aec760b22f-kube-api-access-jz7fr\") pod \"148acf54-c742-4253-b7ae-52aec760b22f\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") "
Aug 03 16:48:41.951897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951789    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-openshift-service-ca\") pod \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") "
Aug 03 16:48:41.951897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951820    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/148acf54-c742-4253-b7ae-52aec760b22f-plugins-config-volume\") pod \"148acf54-c742-4253-b7ae-52aec760b22f\" (UID: \"148acf54-c742-4253-b7ae-52aec760b22f\") "
Aug 03 16:48:41.951897 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.951848    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjvfz\" (UniqueName: \"kubernetes.io/projected/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-kube-api-access-gjvfz\") pod \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\" (UID: \"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce\") "
Aug 03 16:48:41.952227 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.952197    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" (UID: "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:48:41.952227 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.952212    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" (UID: "d9bd61a5-7c2c-4663-bcc1-86dbdc306edf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:48:41.952354 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.952328    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/148acf54-c742-4253-b7ae-52aec760b22f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "148acf54-c742-4253-b7ae-52aec760b22f" (UID: "148acf54-c742-4253-b7ae-52aec760b22f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:48:41.954022 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.953999    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" (UID: "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:48:41.954022 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.954003    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-kube-api-access-gfjdg" (OuterVolumeSpecName: "kube-api-access-gfjdg") pod "d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" (UID: "d9bd61a5-7c2c-4663-bcc1-86dbdc306edf"). InnerVolumeSpecName "kube-api-access-gfjdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:48:41.954413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.954391    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/148acf54-c742-4253-b7ae-52aec760b22f-kube-api-access-jz7fr" (OuterVolumeSpecName: "kube-api-access-jz7fr") pod "148acf54-c742-4253-b7ae-52aec760b22f" (UID: "148acf54-c742-4253-b7ae-52aec760b22f"). InnerVolumeSpecName "kube-api-access-jz7fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:48:41.954462 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:41.954391    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-kube-api-access-gjvfz" (OuterVolumeSpecName: "kube-api-access-gjvfz") pod "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" (UID: "ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce"). InnerVolumeSpecName "kube-api-access-gjvfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:48:42.053406 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053376    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.053406 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053407    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfjdg\" (UniqueName: \"kubernetes.io/projected/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf-kube-api-access-gfjdg\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.053644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053417    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz7fr\" (UniqueName: \"kubernetes.io/projected/148acf54-c742-4253-b7ae-52aec760b22f-kube-api-access-jz7fr\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.053644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053427    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.053644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053436    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/148acf54-c742-4253-b7ae-52aec760b22f-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.053644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053445    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjvfz\" (UniqueName: \"kubernetes.io/projected/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-kube-api-access-gjvfz\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.053644 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.053454    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:42.111276 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.111249    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:42.314474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.314385    2560 generic.go:356] "Generic (PLEG): container finished" podID="148acf54-c742-4253-b7ae-52aec760b22f" containerID="eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707" exitCode=0
Aug 03 16:48:42.314474 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.314437    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"
Aug 03 16:48:42.314701 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.314473    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9" event={"ID":"148acf54-c742-4253-b7ae-52aec760b22f","Type":"ContainerDied","Data":"eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707"}
Aug 03 16:48:42.314701 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.314548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9" event={"ID":"148acf54-c742-4253-b7ae-52aec760b22f","Type":"ContainerDied","Data":"9d7e8b048623b243551176bc480b11b88c46fa9c7e3874ef562462c66229e124"}
Aug 03 16:48:42.314701 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.314578    2560 scope.go:117] "RemoveContainer" containerID="eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707"
Aug 03 16:48:42.316023 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.315990    2560 generic.go:356] "Generic (PLEG): container finished" podID="d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" containerID="2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7" exitCode=0
Aug 03 16:48:42.316155 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.316030    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"
Aug 03 16:48:42.316155 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.316035    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9" event={"ID":"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf","Type":"ContainerDied","Data":"2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7"}
Aug 03 16:48:42.316155 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.316087    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9" event={"ID":"d9bd61a5-7c2c-4663-bcc1-86dbdc306edf","Type":"ContainerDied","Data":"8c9d6d643f07c5bf106e174590632f6b6d3c756329db67216ff25bec91722a26"}
Aug 03 16:48:42.317461 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.317439    2560 generic.go:356] "Generic (PLEG): container finished" podID="ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" containerID="aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df" exitCode=0
Aug 03 16:48:42.317601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.317488    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"
Aug 03 16:48:42.317601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.317488    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" event={"ID":"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce","Type":"ContainerDied","Data":"aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df"}
Aug 03 16:48:42.317601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.317589    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5" event={"ID":"ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce","Type":"ContainerDied","Data":"c11dc27533b70dd2c7de27ed2c0dbf2155a7661884d02aebe51637a68f1f9e22"}
Aug 03 16:48:42.325673 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.325640    2560 scope.go:117] "RemoveContainer" containerID="eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707"
Aug 03 16:48:42.326327 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:48:42.326120    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707\": container with ID starting with eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707 not found: ID does not exist" containerID="eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707"
Aug 03 16:48:42.326327 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.326147    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707"} err="failed to get container status \"eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707\": rpc error: code = NotFound desc = could not find container \"eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707\": container with ID starting with eed1e9f37a9369aeee0e5c8d15b973793459474d1383cfae7d29cff092b28707 not found: ID does not exist"
Aug 03 16:48:42.326327 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.326165    2560 scope.go:117] "RemoveContainer" containerID="2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7"
Aug 03 16:48:42.334684 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.334666    2560 scope.go:117] "RemoveContainer" containerID="2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7"
Aug 03 16:48:42.334915 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:48:42.334895    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7\": container with ID starting with 2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7 not found: ID does not exist" containerID="2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7"
Aug 03 16:48:42.334961 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.334922    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7"} err="failed to get container status \"2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7\": rpc error: code = NotFound desc = could not find container \"2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7\": container with ID starting with 2d79a549de4d46dc4bf99a9093a1882cc623c337ac7319ea085ee1e0345428a7 not found: ID does not exist"
Aug 03 16:48:42.334961 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.334938    2560 scope.go:117] "RemoveContainer" containerID="aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df"
Aug 03 16:48:42.342568 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.342546    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"]
Aug 03 16:48:42.342960 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.342948    2560 scope.go:117] "RemoveContainer" containerID="aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df"
Aug 03 16:48:42.343212 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:48:42.343196    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df\": container with ID starting with aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df not found: ID does not exist" containerID="aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df"
Aug 03 16:48:42.343268 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.343218    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df"} err="failed to get container status \"aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df\": rpc error: code = NotFound desc = could not find container \"aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df\": container with ID starting with aebd295b5d98054ac09cef22abd69b7745fef42a010b725ede5825ad042440df not found: ID does not exist"
Aug 03 16:48:42.349079 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.349059    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-deec1a-5fcb9688cd-p24r9"]
Aug 03 16:48:42.362498 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.362475    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"]
Aug 03 16:48:42.367063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.367042    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-deec1a-989f666d6-xmvt5"]
Aug 03 16:48:42.379116 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.379090    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"]
Aug 03 16:48:42.383596 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.383577    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-deec1a-8bcd566cb-78np9"]
Aug 03 16:48:42.987715 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.987683    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="148acf54-c742-4253-b7ae-52aec760b22f" path="/var/lib/kubelet/pods/148acf54-c742-4253-b7ae-52aec760b22f/volumes"
Aug 03 16:48:42.988097 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.988061    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" path="/var/lib/kubelet/pods/ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce/volumes"
Aug 03 16:48:42.988362 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:42.988349    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" path="/var/lib/kubelet/pods/d9bd61a5-7c2c-4663-bcc1-86dbdc306edf/volumes"
Aug 03 16:48:48.741308 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:48.741267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:49.398265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398216    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"]
Aug 03 16:48:49.398672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398658    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" containerName="maas-api"
Aug 03 16:48:49.398672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398673    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" containerName="maas-api"
Aug 03 16:48:49.398760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398694    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" containerName="payload-pre-processing"
Aug 03 16:48:49.398760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398700    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" containerName="payload-pre-processing"
Aug 03 16:48:49.398760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398712    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="148acf54-c742-4253-b7ae-52aec760b22f" containerName="payload-processing"
Aug 03 16:48:49.398760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398718    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="148acf54-c742-4253-b7ae-52aec760b22f" containerName="payload-processing"
Aug 03 16:48:49.398887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398787    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="148acf54-c742-4253-b7ae-52aec760b22f" containerName="payload-processing"
Aug 03 16:48:49.398887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398796    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9bd61a5-7c2c-4663-bcc1-86dbdc306edf" containerName="payload-pre-processing"
Aug 03 16:48:49.398887 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.398803    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca0fc6b2-ba5c-4379-bd3d-06e55dec49ce" containerName="maas-api"
Aug 03 16:48:49.403543 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.403510    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:49.406301 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.406276    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 16:48:49.410517 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.410486    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"]
Aug 03 16:48:49.516013 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.515982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/751e2d0c-bebe-441a-a053-fd2b608f6d23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk\" (UID: \"751e2d0c-bebe-441a-a053-fd2b608f6d23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:49.551710 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.551677    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:49.613080 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.613044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:48:49.616862 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.616838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/751e2d0c-bebe-441a-a053-fd2b608f6d23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk\" (UID: \"751e2d0c-bebe-441a-a053-fd2b608f6d23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:49.617405 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.617388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/751e2d0c-bebe-441a-a053-fd2b608f6d23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk\" (UID: \"751e2d0c-bebe-441a-a053-fd2b608f6d23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:49.717010 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.716921    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:49.851576 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:49.851551    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"]
Aug 03 16:48:49.854574 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:48:49.854548    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751e2d0c_bebe_441a_a053_fd2b608f6d23.slice/crio-03caf7f61e127b19ea414977fae0213cd8825008722ce8297f35357221fbf615 WatchSource:0}: Error finding container 03caf7f61e127b19ea414977fae0213cd8825008722ce8297f35357221fbf615: Status 404 returned error can't find the container with id 03caf7f61e127b19ea414977fae0213cd8825008722ce8297f35357221fbf615
Aug 03 16:48:50.374033 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:50.373998    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk" event={"ID":"751e2d0c-bebe-441a-a053-fd2b608f6d23","Type":"ContainerStarted","Data":"8697edb15ae5f0c7a5bd8bb0e89a2c9faed6c37733c84085a9e699ced2f1e510"}
Aug 03 16:48:50.374193 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:50.374047    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk" event={"ID":"751e2d0c-bebe-441a-a053-fd2b608f6d23","Type":"ContainerStarted","Data":"03caf7f61e127b19ea414977fae0213cd8825008722ce8297f35357221fbf615"}
Aug 03 16:48:50.391703 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:50.391648    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk" podStartSLOduration=1.3916308499999999 podStartE2EDuration="1.39163085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:48:50.390341571 +0000 UTC m=+2689.939627884" watchObservedRunningTime="2026-08-03 16:48:50.39163085 +0000 UTC m=+2689.940917166"
Aug 03 16:48:51.380386 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:51.380348    2560 generic.go:356] "Generic (PLEG): container finished" podID="751e2d0c-bebe-441a-a053-fd2b608f6d23" containerID="8697edb15ae5f0c7a5bd8bb0e89a2c9faed6c37733c84085a9e699ced2f1e510" exitCode=0
Aug 03 16:48:51.380800 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:51.380422    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk" event={"ID":"751e2d0c-bebe-441a-a053-fd2b608f6d23","Type":"ContainerDied","Data":"8697edb15ae5f0c7a5bd8bb0e89a2c9faed6c37733c84085a9e699ced2f1e510"}
Aug 03 16:48:52.527995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:52.527973    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:52.538284 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:52.538257    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/751e2d0c-bebe-441a-a053-fd2b608f6d23-maas-api-service-ca\") pod \"751e2d0c-bebe-441a-a053-fd2b608f6d23\" (UID: \"751e2d0c-bebe-441a-a053-fd2b608f6d23\") "
Aug 03 16:48:52.538640 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:52.538623    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/751e2d0c-bebe-441a-a053-fd2b608f6d23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "751e2d0c-bebe-441a-a053-fd2b608f6d23" (UID: "751e2d0c-bebe-441a-a053-fd2b608f6d23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:48:52.639302 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:52.639226    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/751e2d0c-bebe-441a-a053-fd2b608f6d23-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:48:53.392191 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:53.392113    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"
Aug 03 16:48:53.392191 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:53.392145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk" event={"ID":"751e2d0c-bebe-441a-a053-fd2b608f6d23","Type":"ContainerDied","Data":"03caf7f61e127b19ea414977fae0213cd8825008722ce8297f35357221fbf615"}
Aug 03 16:48:53.392191 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:48:53.392187    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03caf7f61e127b19ea414977fae0213cd8825008722ce8297f35357221fbf615"
Aug 03 16:49:01.128622 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:01.128594    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:49:01.138366 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:01.138340    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:49:01.377842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:01.377809    2560 scope.go:117] "RemoveContainer" containerID="8275786ffd29044f92b137db3a5f60c5a481ba19c76517c800fd43d265207285"
Aug 03 16:49:05.036951 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.036920    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"]
Aug 03 16:49:05.037422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.037165    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" podUID="8998e4f7-cc82-492b-b93c-8183fdbaab52" containerName="maas-api" containerID="cri-o://398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691" gracePeriod=30
Aug 03 16:49:05.073780 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.073754    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:05.102233 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.102194    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"]
Aug 03 16:49:05.102447 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.102420    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s" podUID="5fc6d07f-2765-48db-a0a4-dad080f08161" containerName="payload-processing" containerID="cri-o://a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b" gracePeriod=30
Aug 03 16:49:05.119948 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.119910    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"]
Aug 03 16:49:05.120269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.120216    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8" podUID="f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" containerName="payload-pre-processing" containerID="cri-o://7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23" gracePeriod=30
Aug 03 16:49:05.190672 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.190601    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:05.285249 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.285191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:05.292404 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.292345    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"]
Aug 03 16:49:05.297059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.296968    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9ffb05-nm8jk"]
Aug 03 16:49:05.385046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.385006    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:05.386982 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.386950    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:49:05.391038 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.391015    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:49:05.424506 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.424480    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:49:05.447105 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447078    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-675d6\" (UniqueName: \"kubernetes.io/projected/5fc6d07f-2765-48db-a0a4-dad080f08161-kube-api-access-675d6\") pod \"5fc6d07f-2765-48db-a0a4-dad080f08161\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") "
Aug 03 16:49:05.447289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447114    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmpf2\" (UniqueName: \"kubernetes.io/projected/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-kube-api-access-qmpf2\") pod \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") "
Aug 03 16:49:05.447289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pxjf\" (UniqueName: \"kubernetes.io/projected/8998e4f7-cc82-492b-b93c-8183fdbaab52-kube-api-access-4pxjf\") pod \"8998e4f7-cc82-492b-b93c-8183fdbaab52\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") "
Aug 03 16:49:05.447289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447176    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8998e4f7-cc82-492b-b93c-8183fdbaab52-openshift-service-ca\") pod \"8998e4f7-cc82-492b-b93c-8183fdbaab52\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") "
Aug 03 16:49:05.447289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447211    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls\") pod \"8998e4f7-cc82-492b-b93c-8183fdbaab52\" (UID: \"8998e4f7-cc82-492b-b93c-8183fdbaab52\") "
Aug 03 16:49:05.447289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447238    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-plugins-config-volume\") pod \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\" (UID: \"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0\") "
Aug 03 16:49:05.447289 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447283    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc6d07f-2765-48db-a0a4-dad080f08161-plugins-config-volume\") pod \"5fc6d07f-2765-48db-a0a4-dad080f08161\" (UID: \"5fc6d07f-2765-48db-a0a4-dad080f08161\") "
Aug 03 16:49:05.447666 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447628    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8998e4f7-cc82-492b-b93c-8183fdbaab52-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8998e4f7-cc82-492b-b93c-8183fdbaab52" (UID: "8998e4f7-cc82-492b-b93c-8183fdbaab52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:49:05.447666 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" (UID: "f724a542-a5f2-4d17-b2c5-5c6d34a75ff0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:49:05.447965 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.447936    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fc6d07f-2765-48db-a0a4-dad080f08161-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5fc6d07f-2765-48db-a0a4-dad080f08161" (UID: "5fc6d07f-2765-48db-a0a4-dad080f08161"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 16:49:05.449735 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.449702    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8998e4f7-cc82-492b-b93c-8183fdbaab52" (UID: "8998e4f7-cc82-492b-b93c-8183fdbaab52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 16:49:05.449837 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.449808    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8998e4f7-cc82-492b-b93c-8183fdbaab52-kube-api-access-4pxjf" (OuterVolumeSpecName: "kube-api-access-4pxjf") pod "8998e4f7-cc82-492b-b93c-8183fdbaab52" (UID: "8998e4f7-cc82-492b-b93c-8183fdbaab52"). InnerVolumeSpecName "kube-api-access-4pxjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:49:05.449953 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.449933    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc6d07f-2765-48db-a0a4-dad080f08161-kube-api-access-675d6" (OuterVolumeSpecName: "kube-api-access-675d6") pod "5fc6d07f-2765-48db-a0a4-dad080f08161" (UID: "5fc6d07f-2765-48db-a0a4-dad080f08161"). InnerVolumeSpecName "kube-api-access-675d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:49:05.450373 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.450346    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-kube-api-access-qmpf2" (OuterVolumeSpecName: "kube-api-access-qmpf2") pod "f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" (UID: "f724a542-a5f2-4d17-b2c5-5c6d34a75ff0"). InnerVolumeSpecName "kube-api-access-qmpf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 16:49:05.460970 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.460941    2560 generic.go:356] "Generic (PLEG): container finished" podID="f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" containerID="7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23" exitCode=0
Aug 03 16:49:05.461085 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.460982    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"
Aug 03 16:49:05.461085 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.461022    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8" event={"ID":"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0","Type":"ContainerDied","Data":"7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23"}
Aug 03 16:49:05.461217 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.461114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8" event={"ID":"f724a542-a5f2-4d17-b2c5-5c6d34a75ff0","Type":"ContainerDied","Data":"fc301a5e07cb7e72b2d6b2eb2b0b584b7fedc15ecab276924ccc284735ab18e1"}
Aug 03 16:49:05.461217 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.461148    2560 scope.go:117] "RemoveContainer" containerID="7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23"
Aug 03 16:49:05.462594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.462515    2560 generic.go:356] "Generic (PLEG): container finished" podID="8998e4f7-cc82-492b-b93c-8183fdbaab52" containerID="398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691" exitCode=0
Aug 03 16:49:05.462594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.462559    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" event={"ID":"8998e4f7-cc82-492b-b93c-8183fdbaab52","Type":"ContainerDied","Data":"398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691"}
Aug 03 16:49:05.462594 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.462578    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"
Aug 03 16:49:05.462817 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.462597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r" event={"ID":"8998e4f7-cc82-492b-b93c-8183fdbaab52","Type":"ContainerDied","Data":"8ebe3886fe187cd23d745d97f8d4ff35cbfddd06c477907152dd20eccfdc6685"}
Aug 03 16:49:05.464138 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.464114    2560 generic.go:356] "Generic (PLEG): container finished" podID="5fc6d07f-2765-48db-a0a4-dad080f08161" containerID="a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b" exitCode=0
Aug 03 16:49:05.464251 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.464162    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"
Aug 03 16:49:05.464251 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.464160    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s" event={"ID":"5fc6d07f-2765-48db-a0a4-dad080f08161","Type":"ContainerDied","Data":"a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b"}
Aug 03 16:49:05.464251 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.464208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s" event={"ID":"5fc6d07f-2765-48db-a0a4-dad080f08161","Type":"ContainerDied","Data":"f7aac694d9d898008ab81da9cc7eca3dc46a52f8ba90e70eb1067a75c79525e1"}
Aug 03 16:49:05.473372 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.473353    2560 scope.go:117] "RemoveContainer" containerID="7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23"
Aug 03 16:49:05.473736 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:05.473715    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23\": container with ID starting with 7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23 not found: ID does not exist" containerID="7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23"
Aug 03 16:49:05.473841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.473745    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23"} err="failed to get container status \"7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23\": rpc error: code = NotFound desc = could not find container \"7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23\": container with ID starting with 7f963c1d46cd7483ee30c8c6ffeaf063d9ea5c82dc52b415a14c168ad423dc23 not found: ID does not exist"
Aug 03 16:49:05.473841 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.473768    2560 scope.go:117] "RemoveContainer" containerID="398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691"
Aug 03 16:49:05.483369 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.483351    2560 scope.go:117] "RemoveContainer" containerID="398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691"
Aug 03 16:49:05.483655 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:05.483638    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691\": container with ID starting with 398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691 not found: ID does not exist" containerID="398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691"
Aug 03 16:49:05.483708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.483661    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691"} err="failed to get container status \"398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691\": rpc error: code = NotFound desc = could not find container \"398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691\": container with ID starting with 398a81491fe422178fdefb70c1cab0aa552c7238becdc8d31b59f1839bfd3691 not found: ID does not exist"
Aug 03 16:49:05.483708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.483674    2560 scope.go:117] "RemoveContainer" containerID="a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b"
Aug 03 16:49:05.490153 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.490130    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"]
Aug 03 16:49:05.492255 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.492237    2560 scope.go:117] "RemoveContainer" containerID="a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b"
Aug 03 16:49:05.492422 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.492407    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9ffb05-7bf6594dd-ckbt8"]
Aug 03 16:49:05.492522 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:05.492504    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b\": container with ID starting with a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b not found: ID does not exist" containerID="a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b"
Aug 03 16:49:05.492618 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.492548    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b"} err="failed to get container status \"a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b\": rpc error: code = NotFound desc = could not find container \"a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b\": container with ID starting with a420da7ce27834118c8d0e03c7f440168d69b170635c0733430e8de06dd89e5b not found: ID does not exist"
Aug 03 16:49:05.505107 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.505085    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"]
Aug 03 16:49:05.509842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.509818    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9ffb05-64b47d8cc4-8q22s"]
Aug 03 16:49:05.520267 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.520243    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"]
Aug 03 16:49:05.526351 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.526330    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9ffb05-65446c68c6-rhz8r"]
Aug 03 16:49:05.549117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549043    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8998e4f7-cc82-492b-b93c-8183fdbaab52-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.549117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549069    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8998e4f7-cc82-492b-b93c-8183fdbaab52-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.549117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549085    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.549117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549096    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc6d07f-2765-48db-a0a4-dad080f08161-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.549117 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549108    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-675d6\" (UniqueName: \"kubernetes.io/projected/5fc6d07f-2765-48db-a0a4-dad080f08161-kube-api-access-675d6\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.549345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549121    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmpf2\" (UniqueName: \"kubernetes.io/projected/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0-kube-api-access-qmpf2\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.549345 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.549133    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pxjf\" (UniqueName: \"kubernetes.io/projected/8998e4f7-cc82-492b-b93c-8183fdbaab52-kube-api-access-4pxjf\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 16:49:05.697027 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:05.696991    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:06.987152 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:06.987119    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc6d07f-2765-48db-a0a4-dad080f08161" path="/var/lib/kubelet/pods/5fc6d07f-2765-48db-a0a4-dad080f08161/volumes"
Aug 03 16:49:06.987556 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:06.987449    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="751e2d0c-bebe-441a-a053-fd2b608f6d23" path="/var/lib/kubelet/pods/751e2d0c-bebe-441a-a053-fd2b608f6d23/volumes"
Aug 03 16:49:06.987799 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:06.987786    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8998e4f7-cc82-492b-b93c-8183fdbaab52" path="/var/lib/kubelet/pods/8998e4f7-cc82-492b-b93c-8183fdbaab52/volumes"
Aug 03 16:49:06.988086 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:06.988075    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" path="/var/lib/kubelet/pods/f724a542-a5f2-4d17-b2c5-5c6d34a75ff0/volumes"
Aug 03 16:49:12.052280 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:12.052249    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:12.123221 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:12.123188    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:12.327549 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:12.327439    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:12.611114 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:12.611035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:12.910146 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:12.910069    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:13.227608 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:13.227492    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:13.305063 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:13.305030    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:16.721502 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:16.721469    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:16.822158 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:16.822127    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:17.833187 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833147    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"]
Aug 03 16:49:17.833756 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833740    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8998e4f7-cc82-492b-b93c-8183fdbaab52" containerName="maas-api"
Aug 03 16:49:17.833810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833760    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8998e4f7-cc82-492b-b93c-8183fdbaab52" containerName="maas-api"
Aug 03 16:49:17.833810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833785    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751e2d0c-bebe-441a-a053-fd2b608f6d23" containerName="revoke-keys"
Aug 03 16:49:17.833810 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833793    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="751e2d0c-bebe-441a-a053-fd2b608f6d23" containerName="revoke-keys"
Aug 03 16:49:17.833912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833816    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc6d07f-2765-48db-a0a4-dad080f08161" containerName="payload-processing"
Aug 03 16:49:17.833912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833825    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc6d07f-2765-48db-a0a4-dad080f08161" containerName="payload-processing"
Aug 03 16:49:17.833912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833841    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" containerName="payload-pre-processing"
Aug 03 16:49:17.833912 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833850    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" containerName="payload-pre-processing"
Aug 03 16:49:17.834041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833937    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f724a542-a5f2-4d17-b2c5-5c6d34a75ff0" containerName="payload-pre-processing"
Aug 03 16:49:17.834041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833950    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="751e2d0c-bebe-441a-a053-fd2b608f6d23" containerName="revoke-keys"
Aug 03 16:49:17.834041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833960    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fc6d07f-2765-48db-a0a4-dad080f08161" containerName="payload-processing"
Aug 03 16:49:17.834041 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.833971    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8998e4f7-cc82-492b-b93c-8183fdbaab52" containerName="maas-api"
Aug 03 16:49:17.837506 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.837485    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:17.840402 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.840379    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0fae8e-serving-cert\""
Aug 03 16:49:17.848511 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.848477    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"]
Aug 03 16:49:17.875093 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.875063    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:17.966044 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.966007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:17.966246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.966080    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whr67\" (UniqueName: \"kubernetes.io/projected/f4d52db1-d273-40b1-9793-d4af1b53e77e-kube-api-access-whr67\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:17.966246 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:17.966152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4d52db1-d273-40b1-9793-d4af1b53e77e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.066888 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.066851    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.067102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.066919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whr67\" (UniqueName: \"kubernetes.io/projected/f4d52db1-d273-40b1-9793-d4af1b53e77e-kube-api-access-whr67\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.067102 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.066981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4d52db1-d273-40b1-9793-d4af1b53e77e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.067102 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:18.067014    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-serving-cert: secret "maas-api-e2e-shared-a-0fae8e-serving-cert" not found
Aug 03 16:49:18.067102 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:18.067087    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls podName:f4d52db1-d273-40b1-9793-d4af1b53e77e nodeName:}" failed. No retries permitted until 2026-08-03 16:49:18.567064421 +0000 UTC m=+2718.116350722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls") pod "maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" (UID: "f4d52db1-d273-40b1-9793-d4af1b53e77e") : secret "maas-api-e2e-shared-a-0fae8e-serving-cert" not found
Aug 03 16:49:18.067664 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.067643    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4d52db1-d273-40b1-9793-d4af1b53e77e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.078213 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.078187    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whr67\" (UniqueName: \"kubernetes.io/projected/f4d52db1-d273-40b1-9793-d4af1b53e77e-kube-api-access-whr67\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.150131 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.150038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:18.369454 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.369423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:18.467736 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.467652    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:18.477473 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.477447    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"]
Aug 03 16:49:18.481030 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.481014    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.483971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.483948    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-0fae8e\""
Aug 03 16:49:18.484116 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.484100    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-0fae8e-dockercfg-dgsxv\""
Aug 03 16:49:18.494505 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.494480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"]
Aug 03 16:49:18.502464 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.502433    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"]
Aug 03 16:49:18.505867 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.505850    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.525618 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.525590    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"]
Aug 03 16:49:18.571323 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.571288    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61a37f6-12e6-410a-8f01-74e7aecb5f39-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.571513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.571353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzsl2\" (UniqueName: \"kubernetes.io/projected/736ecbd0-5e7f-4e7b-b951-632c189706fd-kube-api-access-zzsl2\") pod \"payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.571513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.571432    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.571513 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.571488    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwb5l\" (UniqueName: \"kubernetes.io/projected/c61a37f6-12e6-410a-8f01-74e7aecb5f39-kube-api-access-rwb5l\") pod \"payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.571726 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.571522    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/736ecbd0-5e7f-4e7b-b951-632c189706fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.574247 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.574212    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.672958 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.672916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwb5l\" (UniqueName: \"kubernetes.io/projected/c61a37f6-12e6-410a-8f01-74e7aecb5f39-kube-api-access-rwb5l\") pod \"payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.672958 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.672968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/736ecbd0-5e7f-4e7b-b951-632c189706fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.673192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.672994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61a37f6-12e6-410a-8f01-74e7aecb5f39-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.673192 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.673027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzsl2\" (UniqueName: \"kubernetes.io/projected/736ecbd0-5e7f-4e7b-b951-632c189706fd-kube-api-access-zzsl2\") pod \"payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.673777 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.673757    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61a37f6-12e6-410a-8f01-74e7aecb5f39-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.673842 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.673782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/736ecbd0-5e7f-4e7b-b951-632c189706fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.682458 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.682422    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwb5l\" (UniqueName: \"kubernetes.io/projected/c61a37f6-12e6-410a-8f01-74e7aecb5f39-kube-api-access-rwb5l\") pod \"payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.682640 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.682600    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzsl2\" (UniqueName: \"kubernetes.io/projected/736ecbd0-5e7f-4e7b-b951-632c189706fd-kube-api-access-zzsl2\") pod \"payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.749755 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.749661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:18.792256 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.792225    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:18.816285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.816255    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:18.942013 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:18.941971    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"]
Aug 03 16:49:18.944623 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:49:18.944590    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d52db1_d273_40b1_9793_d4af1b53e77e.slice/crio-e129fffa955ce4c5d355505f06c899bbfe8f40159541e6bc3d842f9e6b2adad1 WatchSource:0}: Error finding container e129fffa955ce4c5d355505f06c899bbfe8f40159541e6bc3d842f9e6b2adad1: Status 404 returned error can't find the container with id e129fffa955ce4c5d355505f06c899bbfe8f40159541e6bc3d842f9e6b2adad1
Aug 03 16:49:19.001915 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.001890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"]
Aug 03 16:49:19.005287 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:49:19.005256    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61a37f6_12e6_410a_8f01_74e7aecb5f39.slice/crio-871083835d26f7cee974382570d93edeb0452370f482917f8de972330e431af5 WatchSource:0}: Error finding container 871083835d26f7cee974382570d93edeb0452370f482917f8de972330e431af5: Status 404 returned error can't find the container with id 871083835d26f7cee974382570d93edeb0452370f482917f8de972330e431af5
Aug 03 16:49:19.184916 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.184890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"]
Aug 03 16:49:19.186423 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:49:19.186384    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736ecbd0_5e7f_4e7b_b951_632c189706fd.slice/crio-d01f92d7b69b9edf9a03a2817aa5c49e14a73b30ad1e6e0fd0fedd8a83a74933 WatchSource:0}: Error finding container d01f92d7b69b9edf9a03a2817aa5c49e14a73b30ad1e6e0fd0fedd8a83a74933: Status 404 returned error can't find the container with id d01f92d7b69b9edf9a03a2817aa5c49e14a73b30ad1e6e0fd0fedd8a83a74933
Aug 03 16:49:19.540190 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.540159    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" event={"ID":"f4d52db1-d273-40b1-9793-d4af1b53e77e","Type":"ContainerStarted","Data":"2f796466664325cbc214933f6946b093120421c86f21d92ae1e72ec73a71b5b3"}
Aug 03 16:49:19.540365 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.540218    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" event={"ID":"f4d52db1-d273-40b1-9793-d4af1b53e77e","Type":"ContainerStarted","Data":"e129fffa955ce4c5d355505f06c899bbfe8f40159541e6bc3d842f9e6b2adad1"}
Aug 03 16:49:19.540365 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.540276    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:19.541971 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.541940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8" event={"ID":"736ecbd0-5e7f-4e7b-b951-632c189706fd","Type":"ContainerStarted","Data":"a81235888fdef6538b5a7f3e13ca04d4d7b6cca708e7cb4467a2b2e17590cea6"}
Aug 03 16:49:19.542089 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.541990    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8" event={"ID":"736ecbd0-5e7f-4e7b-b951-632c189706fd","Type":"ContainerStarted","Data":"d01f92d7b69b9edf9a03a2817aa5c49e14a73b30ad1e6e0fd0fedd8a83a74933"}
Aug 03 16:49:19.542089 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.542019    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:19.543642 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.543619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5" event={"ID":"c61a37f6-12e6-410a-8f01-74e7aecb5f39","Type":"ContainerStarted","Data":"c85ea1258163590c74856abdf3901bfe1698806df588cd58f0e0f56665420a1f"}
Aug 03 16:49:19.543758 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.543666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5" event={"ID":"c61a37f6-12e6-410a-8f01-74e7aecb5f39","Type":"ContainerStarted","Data":"871083835d26f7cee974382570d93edeb0452370f482917f8de972330e431af5"}
Aug 03 16:49:19.543758 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.543694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:19.561770 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.561715    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" podStartSLOduration=2.038811099 podStartE2EDuration="2.561698495s" totalImagesPullingTime="522.887396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:19.560855298 +0000 UTC m=+2719.110141615" watchObservedRunningTime="2026-08-03 16:49:19.561698495 +0000 UTC m=+2719.110984811"
Aug 03 16:49:19.581074 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.581026    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8" podStartSLOduration=1.5810103660000001 podStartE2EDuration="1.581010366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:19.580359274 +0000 UTC m=+2719.129645591" watchObservedRunningTime="2026-08-03 16:49:19.581010366 +0000 UTC m=+2719.130296681"
Aug 03 16:49:19.600144 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:19.599266    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5" podStartSLOduration=1.599243277 podStartE2EDuration="1.599243277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:19.597345887 +0000 UTC m=+2719.146632196" watchObservedRunningTime="2026-08-03 16:49:19.599243277 +0000 UTC m=+2719.148529597"
Aug 03 16:49:25.557915 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:25.557880    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 16:49:30.553673 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:30.553640    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 16:49:30.554067 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:30.553825    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 16:49:41.773504 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:41.773464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:42.122515 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:42.122466    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:42.515980 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:42.515945    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:42.772850 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:42.772765    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:46.065622 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:46.065592    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:46.234263 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:46.234222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:47.379829 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.379792    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:47.383210 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.383182    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"]
Aug 03 16:49:47.387464 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.387410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.391059 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.391038    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-7055f0-serving-cert\""
Aug 03 16:49:47.406111 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.406080    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"]
Aug 03 16:49:47.426099 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.426069    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.426267 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.426150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df64002-fab9-41da-8eb5-033d08ac228e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.426267 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.426209    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlvlv\" (UniqueName: \"kubernetes.io/projected/9df64002-fab9-41da-8eb5-033d08ac228e-kube-api-access-jlvlv\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.526627 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.526588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlvlv\" (UniqueName: \"kubernetes.io/projected/9df64002-fab9-41da-8eb5-033d08ac228e-kube-api-access-jlvlv\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.526839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.526681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.526839 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.526743    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df64002-fab9-41da-8eb5-033d08ac228e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.526839 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:47.526823    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-serving-cert: secret "maas-api-e2e-shared-b-7055f0-serving-cert" not found
Aug 03 16:49:47.527012 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:49:47.526915    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls podName:9df64002-fab9-41da-8eb5-033d08ac228e nodeName:}" failed. No retries permitted until 2026-08-03 16:49:48.026898594 +0000 UTC m=+2747.576184893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls") pod "maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" (UID: "9df64002-fab9-41da-8eb5-033d08ac228e") : secret "maas-api-e2e-shared-b-7055f0-serving-cert" not found
Aug 03 16:49:47.527403 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.527380    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df64002-fab9-41da-8eb5-033d08ac228e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.548378 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.548345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlvlv\" (UniqueName: \"kubernetes.io/projected/9df64002-fab9-41da-8eb5-033d08ac228e-kube-api-access-jlvlv\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:47.895167 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:47.895117    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:48.030549 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.030484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:48.033031 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.033008    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls\") pod \"maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:48.080012 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.079959    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:48.194635 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.194542    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:49:48.294626 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.294590    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"]
Aug 03 16:49:48.298341 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.298318    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.302783 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.302758    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:48.303732 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.303714    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-7055f0-dockercfg-kl48v\""
Aug 03 16:49:48.305972 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.305771    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-7055f0\""
Aug 03 16:49:48.312648 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.312621    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"]
Aug 03 16:49:48.333983 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.333948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d894c9e-ca69-4931-8184-a792d69d2e6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.334148 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.334082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnfxg\" (UniqueName: \"kubernetes.io/projected/6d894c9e-ca69-4931-8184-a792d69d2e6c-kube-api-access-rnfxg\") pod \"payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.382199 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.382126    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"]
Aug 03 16:49:48.392650 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.392515    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.403546 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.403504    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"]
Aug 03 16:49:48.436462 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.436420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0118505a-4d26-4cfd-b95a-32e5dbd3927b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.436662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.436468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d894c9e-ca69-4931-8184-a792d69d2e6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.436662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.436556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnfxg\" (UniqueName: \"kubernetes.io/projected/6d894c9e-ca69-4931-8184-a792d69d2e6c-kube-api-access-rnfxg\") pod \"payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.436662 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.436579    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf9j7\" (UniqueName: \"kubernetes.io/projected/0118505a-4d26-4cfd-b95a-32e5dbd3927b-kube-api-access-bf9j7\") pod \"payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.437221 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.437199    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d894c9e-ca69-4931-8184-a792d69d2e6c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.453194 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.453122    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnfxg\" (UniqueName: \"kubernetes.io/projected/6d894c9e-ca69-4931-8184-a792d69d2e6c-kube-api-access-rnfxg\") pod \"payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.493687 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.493657    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"]
Aug 03 16:49:48.496336 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:49:48.496304    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df64002_fab9_41da_8eb5_033d08ac228e.slice/crio-0d1a74234ecb484b917a7375405eac5cdeb5bb481304333640d69899ad3f630e WatchSource:0}: Error finding container 0d1a74234ecb484b917a7375405eac5cdeb5bb481304333640d69899ad3f630e: Status 404 returned error can't find the container with id 0d1a74234ecb484b917a7375405eac5cdeb5bb481304333640d69899ad3f630e
Aug 03 16:49:48.537144 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.537111    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf9j7\" (UniqueName: \"kubernetes.io/projected/0118505a-4d26-4cfd-b95a-32e5dbd3927b-kube-api-access-bf9j7\") pod \"payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.537285 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.537167    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0118505a-4d26-4cfd-b95a-32e5dbd3927b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.538212 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.538179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0118505a-4d26-4cfd-b95a-32e5dbd3927b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.551234 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.551206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf9j7\" (UniqueName: \"kubernetes.io/projected/0118505a-4d26-4cfd-b95a-32e5dbd3927b-kube-api-access-bf9j7\") pod \"payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.609202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.609168    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:48.699601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.699564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" event={"ID":"9df64002-fab9-41da-8eb5-033d08ac228e","Type":"ContainerStarted","Data":"0d1a74234ecb484b917a7375405eac5cdeb5bb481304333640d69899ad3f630e"}
Aug 03 16:49:48.710989 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.710897    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:48.786649 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.786509    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"]
Aug 03 16:49:48.788578 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:49:48.788522    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d894c9e_ca69_4931_8184_a792d69d2e6c.slice/crio-582638b486342cfd091e3318191f9c326c319c41be7c859be604203dd2db20da WatchSource:0}: Error finding container 582638b486342cfd091e3318191f9c326c319c41be7c859be604203dd2db20da: Status 404 returned error can't find the container with id 582638b486342cfd091e3318191f9c326c319c41be7c859be604203dd2db20da
Aug 03 16:49:48.869312 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:48.869283    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"]
Aug 03 16:49:48.873806 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:49:48.873756    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0118505a_4d26_4cfd_b95a_32e5dbd3927b.slice/crio-0eb1c7c25da05b593f57ecdc521b0e1878d96d12287dee8d660daeff669ac2ae WatchSource:0}: Error finding container 0eb1c7c25da05b593f57ecdc521b0e1878d96d12287dee8d660daeff669ac2ae: Status 404 returned error can't find the container with id 0eb1c7c25da05b593f57ecdc521b0e1878d96d12287dee8d660daeff669ac2ae
Aug 03 16:49:49.707978 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.707931    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99" event={"ID":"6d894c9e-ca69-4931-8184-a792d69d2e6c","Type":"ContainerStarted","Data":"8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52"}
Aug 03 16:49:49.708448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.708019    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99" event={"ID":"6d894c9e-ca69-4931-8184-a792d69d2e6c","Type":"ContainerStarted","Data":"582638b486342cfd091e3318191f9c326c319c41be7c859be604203dd2db20da"}
Aug 03 16:49:49.708448 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.708059    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:49:49.709929 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.709904    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" event={"ID":"9df64002-fab9-41da-8eb5-033d08ac228e","Type":"ContainerStarted","Data":"0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9"}
Aug 03 16:49:49.710076 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.709964    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:49:49.711419 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.711393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm" event={"ID":"0118505a-4d26-4cfd-b95a-32e5dbd3927b","Type":"ContainerStarted","Data":"e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7"}
Aug 03 16:49:49.711520 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.711432    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm" event={"ID":"0118505a-4d26-4cfd-b95a-32e5dbd3927b","Type":"ContainerStarted","Data":"0eb1c7c25da05b593f57ecdc521b0e1878d96d12287dee8d660daeff669ac2ae"}
Aug 03 16:49:49.711520 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.711464    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:49:49.766025 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.765971    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99" podStartSLOduration=1.765951949 podStartE2EDuration="1.765951949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:49.737716 +0000 UTC m=+2749.287002316" watchObservedRunningTime="2026-08-03 16:49:49.765951949 +0000 UTC m=+2749.315238264"
Aug 03 16:49:49.766595 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.766562    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" podStartSLOduration=2.41843306 podStartE2EDuration="2.76654419s" totalImagesPullingTime="348.11113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:49.76624835 +0000 UTC m=+2749.315534667" watchObservedRunningTime="2026-08-03 16:49:49.76654419 +0000 UTC m=+2749.315830499"
Aug 03 16:49:49.794813 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:49.794749    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm" podStartSLOduration=1.794728542 podStartE2EDuration="1.794728542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:49.791613734 +0000 UTC m=+2749.340900050" watchObservedRunningTime="2026-08-03 16:49:49.794728542 +0000 UTC m=+2749.344014860"
Aug 03 16:49:55.722400 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:49:55.722369    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 16:50:00.718764 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:50:00.718714    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 16:50:00.719231 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:50:00.718834    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 16:50:11.062833 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:50:11.062801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:50:16.903980 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:50:16.903946    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:50:18.463332 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:50:18.463297    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:50:43.786365 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:50:43.786330    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:51:01.430991 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:51:01.430957    2560 scope.go:117] "RemoveContainer" containerID="8c0547ce0e2d16d5052adda4831debf5c7e6792273054707847fd6c8990adaef"
Aug 03 16:51:23.640745 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:51:23.640711    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:52:55.420785 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:52:55.420751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:53:01.459434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:53:01.459399    2560 scope.go:117] "RemoveContainer" containerID="7cbc6ffd685d804f412d86215058c683b15df15d10bfe2026574098b5722cf64"
Aug 03 16:53:18.050457 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:53:18.050409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:54:01.160739 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:54:01.160711    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:54:01.172510 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:54:01.172476    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:54:01.485069 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:54:01.484978    2560 scope.go:117] "RemoveContainer" containerID="8be9b87f207e9c8193c2d5d76db23c2b777dc7b1e1bf8f8b41d865f7f20e9522"
Aug 03 16:54:23.203564 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:54:23.203504    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:54:24.173152 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:54:24.173117    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:54:38.477854 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:54:38.477820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:55:01.506150 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:55:01.506042    2560 scope.go:117] "RemoveContainer" containerID="8697edb15ae5f0c7a5bd8bb0e89a2c9faed6c37733c84085a9e699ced2f1e510"
Aug 03 16:55:08.666955 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:55:08.666871    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:55:49.732191 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:55:49.732152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:55:53.362277 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:55:53.362235    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:55:53.462185 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:55:53.462157    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:56:22.177125 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:56:22.177039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:56:23.060967 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:56:23.060922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:56:33.180658 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:56:33.180620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:56:48.684745 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:56:48.684708    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:56:48.757997 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:56:48.757965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:56:52.385341 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:56:52.385304    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:22.202989 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:22.202891    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:25.942976 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:25.942938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:26.765713 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:26.765679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:47.530001 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:47.529964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:47.631493 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:47.631464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:50.018851 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:50.018814    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:50.111778 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:50.111741    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:51.259060 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.259024    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"]
Aug 03 16:58:51.263315 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.263286    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.265790 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.265772    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0e4b78-serving-cert\""
Aug 03 16:58:51.274423 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.274398    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"]
Aug 03 16:58:51.303538 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.303496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:51.390451 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.390422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3125fdb7-e35d-40bd-8bad-2aa82a750015-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.390646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.390489    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvnnk\" (UniqueName: \"kubernetes.io/projected/3125fdb7-e35d-40bd-8bad-2aa82a750015-kube-api-access-qvnnk\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.390646 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.390541    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3125fdb7-e35d-40bd-8bad-2aa82a750015-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.393166 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.393128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:51.492052 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.492020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvnnk\" (UniqueName: \"kubernetes.io/projected/3125fdb7-e35d-40bd-8bad-2aa82a750015-kube-api-access-qvnnk\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.492226 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.492058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3125fdb7-e35d-40bd-8bad-2aa82a750015-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.492226 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.492150    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3125fdb7-e35d-40bd-8bad-2aa82a750015-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.492760 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.492730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3125fdb7-e35d-40bd-8bad-2aa82a750015-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.494704    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3125fdb7-e35d-40bd-8bad-2aa82a750015-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.501027 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.501000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvnnk\" (UniqueName: \"kubernetes.io/projected/3125fdb7-e35d-40bd-8bad-2aa82a750015-kube-api-access-qvnnk\") pod \"maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.576387 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.576359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:51.721969 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.721942    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"]
Aug 03 16:58:51.723246 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:58:51.723220    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3125fdb7_e35d_40bd_8bad_2aa82a750015.slice/crio-f4d8b79c7183313935bf5ac0683a3c166278306cb66e0f7ea449f6d54e2e1c88 WatchSource:0}: Error finding container f4d8b79c7183313935bf5ac0683a3c166278306cb66e0f7ea449f6d54e2e1c88: Status 404 returned error can't find the container with id f4d8b79c7183313935bf5ac0683a3c166278306cb66e0f7ea449f6d54e2e1c88
Aug 03 16:58:51.724947 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.724930    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 03 16:58:51.800202 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.800099    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:51.895329 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.895247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:58:51.905403 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.905379    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"]
Aug 03 16:58:51.910269 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.910250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:51.913401 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.913375    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-0e4b78\""
Aug 03 16:58:51.919199 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.919175    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"]
Aug 03 16:58:51.928440 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.928420    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"]
Aug 03 16:58:51.931950 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.931932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:51.935275 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.935258    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-0e4b78-dockercfg-9wvfc\""
Aug 03 16:58:51.943434 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.943413    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"]
Aug 03 16:58:51.998114 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.998081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52452a70-45fd-42b2-9ae1-8ffdffaaab12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:51.998287 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:51.998140    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfsql\" (UniqueName: \"kubernetes.io/projected/52452a70-45fd-42b2-9ae1-8ffdffaaab12-kube-api-access-zfsql\") pod \"payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.099207 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.099167    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfsql\" (UniqueName: \"kubernetes.io/projected/52452a70-45fd-42b2-9ae1-8ffdffaaab12-kube-api-access-zfsql\") pod \"payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.099405 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.099273    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tcm2\" (UniqueName: \"kubernetes.io/projected/34eba44b-645f-48d4-8f6e-c9c1866401e0-kube-api-access-4tcm2\") pod \"payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.099516 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.099488    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34eba44b-645f-48d4-8f6e-c9c1866401e0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.099601 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.099576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52452a70-45fd-42b2-9ae1-8ffdffaaab12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.100319 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.100297    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52452a70-45fd-42b2-9ae1-8ffdffaaab12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.108419 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.108397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfsql\" (UniqueName: \"kubernetes.io/projected/52452a70-45fd-42b2-9ae1-8ffdffaaab12-kube-api-access-zfsql\") pod \"payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.200995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.200918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tcm2\" (UniqueName: \"kubernetes.io/projected/34eba44b-645f-48d4-8f6e-c9c1866401e0-kube-api-access-4tcm2\") pod \"payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.200995 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.200969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34eba44b-645f-48d4-8f6e-c9c1866401e0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.201612 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.201595    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34eba44b-645f-48d4-8f6e-c9c1866401e0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.209786 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.209764    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tcm2\" (UniqueName: \"kubernetes.io/projected/34eba44b-645f-48d4-8f6e-c9c1866401e0-kube-api-access-4tcm2\") pod \"payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.222616 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.222597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.245773 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.245752    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:52.379863 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.379830    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"]
Aug 03 16:58:52.380969 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:58:52.380938    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52452a70_45fd_42b2_9ae1_8ffdffaaab12.slice/crio-3a56d489c9576e1cbb2f461f6f9f15d04840fe37752b9dc4841b88f6158662ff WatchSource:0}: Error finding container 3a56d489c9576e1cbb2f461f6f9f15d04840fe37752b9dc4841b88f6158662ff: Status 404 returned error can't find the container with id 3a56d489c9576e1cbb2f461f6f9f15d04840fe37752b9dc4841b88f6158662ff
Aug 03 16:58:52.537739 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.537699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn" event={"ID":"3125fdb7-e35d-40bd-8bad-2aa82a750015","Type":"ContainerStarted","Data":"f4d8b79c7183313935bf5ac0683a3c166278306cb66e0f7ea449f6d54e2e1c88"}
Aug 03 16:58:52.539265 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.539239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz" event={"ID":"52452a70-45fd-42b2-9ae1-8ffdffaaab12","Type":"ContainerStarted","Data":"2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3"}
Aug 03 16:58:52.539396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.539299    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:58:52.539396 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.539324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz" event={"ID":"52452a70-45fd-42b2-9ae1-8ffdffaaab12","Type":"ContainerStarted","Data":"3a56d489c9576e1cbb2f461f6f9f15d04840fe37752b9dc4841b88f6158662ff"}
Aug 03 16:58:52.558708 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.558661    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz" podStartSLOduration=1.5586463579999998 podStartE2EDuration="1.558646358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:58:52.556585245 +0000 UTC m=+3292.105871564" watchObservedRunningTime="2026-08-03 16:58:52.558646358 +0000 UTC m=+3292.107932673"
Aug 03 16:58:52.611040 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:52.611015    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"]
Aug 03 16:58:52.614165 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:58:52.614103    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34eba44b_645f_48d4_8f6e_c9c1866401e0.slice/crio-d519fc9a55b2c9ee0af642e2e345939fea5174ecff4ff1daa9b450a2956972ae WatchSource:0}: Error finding container d519fc9a55b2c9ee0af642e2e345939fea5174ecff4ff1daa9b450a2956972ae: Status 404 returned error can't find the container with id d519fc9a55b2c9ee0af642e2e345939fea5174ecff4ff1daa9b450a2956972ae
Aug 03 16:58:53.546086 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.546045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp" event={"ID":"34eba44b-645f-48d4-8f6e-c9c1866401e0","Type":"ContainerStarted","Data":"1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c"}
Aug 03 16:58:53.546586 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.546109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp" event={"ID":"34eba44b-645f-48d4-8f6e-c9c1866401e0","Type":"ContainerStarted","Data":"d519fc9a55b2c9ee0af642e2e345939fea5174ecff4ff1daa9b450a2956972ae"}
Aug 03 16:58:53.546586 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.546139    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:58:53.547659 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.547636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn" event={"ID":"3125fdb7-e35d-40bd-8bad-2aa82a750015","Type":"ContainerStarted","Data":"76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e"}
Aug 03 16:58:53.547881 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.547865    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:58:53.565484 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.565438    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp" podStartSLOduration=2.5654268350000002 podStartE2EDuration="2.565426835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:58:53.563272005 +0000 UTC m=+3293.112558350" watchObservedRunningTime="2026-08-03 16:58:53.565426835 +0000 UTC m=+3293.114713151"
Aug 03 16:58:53.582582 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:53.582539    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn" podStartSLOduration=1.441773288 podStartE2EDuration="2.582514909s" totalImagesPullingTime="1.140741621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:58:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:58:53.58038259 +0000 UTC m=+3293.129668911" watchObservedRunningTime="2026-08-03 16:58:53.582514909 +0000 UTC m=+3293.131801224"
Aug 03 16:58:59.558763 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:58:59.558735    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 16:59:01.193159 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:01.193129    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:59:01.210372 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:01.210343    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 16:59:03.550522 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:03.550476    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 16:59:04.555105 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:04.555071    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 16:59:15.007587 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:15.007513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:15.728164 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:15.728126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:16.053725 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:16.053688    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:16.233096 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:16.233064    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:19.180366 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:19.180332    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:19.282050 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:19.282015    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:19.911611 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:19.911571    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:20.272244 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.272153    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"]
Aug 03 16:59:20.276227 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.276207    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.279002 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.278978    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-cf8ed9-serving-cert\""
Aug 03 16:59:20.285744 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.285721    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"]
Aug 03 16:59:20.303695 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.303653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:20.371337 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.371288    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.371560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.371355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgg4\" (UniqueName: \"kubernetes.io/projected/500419f8-9c90-41f9-bb7b-fcdac23b752e-kube-api-access-mcgg4\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.371560 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.371497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/500419f8-9c90-41f9-bb7b-fcdac23b752e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.472957 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.472917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/500419f8-9c90-41f9-bb7b-fcdac23b752e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.473175 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.472998    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.473175 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.473035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgg4\" (UniqueName: \"kubernetes.io/projected/500419f8-9c90-41f9-bb7b-fcdac23b752e-kube-api-access-mcgg4\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.473175 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:59:20.473147    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-serving-cert: secret "maas-api-e2e-ipp-b-cf8ed9-serving-cert" not found
Aug 03 16:59:20.473353 ip-10-0-133-47 kubenswrapper[2560]: E0803 16:59:20.473244    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls podName:500419f8-9c90-41f9-bb7b-fcdac23b752e nodeName:}" failed. No retries permitted until 2026-08-03 16:59:20.973219871 +0000 UTC m=+3320.522506186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls") pod "maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" (UID: "500419f8-9c90-41f9-bb7b-fcdac23b752e") : secret "maas-api-e2e-ipp-b-cf8ed9-serving-cert" not found
Aug 03 16:59:20.473690 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.473671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/500419f8-9c90-41f9-bb7b-fcdac23b752e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.482163 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.482140    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgg4\" (UniqueName: \"kubernetes.io/projected/500419f8-9c90-41f9-bb7b-fcdac23b752e-kube-api-access-mcgg4\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.799264 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.799227    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:20.903653 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.903614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:20.904944 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.904922    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"]
Aug 03 16:59:20.908768 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.908749    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:20.911840 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.911817    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-cf8ed9\""
Aug 03 16:59:20.917119 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.917093    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"]
Aug 03 16:59:20.977942 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.977904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:20.978145 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.977961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ba03ed-acf9-4658-88bb-f880cfe3f129-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:20.978145 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.977999    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqd49\" (UniqueName: \"kubernetes.io/projected/11ba03ed-acf9-4658-88bb-f880cfe3f129-kube-api-access-kqd49\") pod \"payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:20.980444 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:20.980410    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:21.079242 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.079147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqd49\" (UniqueName: \"kubernetes.io/projected/11ba03ed-acf9-4658-88bb-f880cfe3f129-kube-api-access-kqd49\") pod \"payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:21.079413 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.079287    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ba03ed-acf9-4658-88bb-f880cfe3f129-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:21.080047 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.080027    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ba03ed-acf9-4658-88bb-f880cfe3f129-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:21.089048 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.089020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqd49\" (UniqueName: \"kubernetes.io/projected/11ba03ed-acf9-4658-88bb-f880cfe3f129-kube-api-access-kqd49\") pod \"payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:21.189198 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.189142    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:21.221564 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.221502    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:21.352508 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.352472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"]
Aug 03 16:59:21.355169 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:59:21.355102    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500419f8_9c90_41f9_bb7b_fcdac23b752e.slice/crio-0ecc8565c3dfa535f8b1a57552a8514f48009a25eb27996c5a6550b8811636ed WatchSource:0}: Error finding container 0ecc8565c3dfa535f8b1a57552a8514f48009a25eb27996c5a6550b8811636ed: Status 404 returned error can't find the container with id 0ecc8565c3dfa535f8b1a57552a8514f48009a25eb27996c5a6550b8811636ed
Aug 03 16:59:21.393046 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.393017    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"]
Aug 03 16:59:21.395811 ip-10-0-133-47 kubenswrapper[2560]: W0803 16:59:21.395776    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ba03ed_acf9_4658_88bb_f880cfe3f129.slice/crio-dce5e2de1885ccd8f1c972fd1c5ab39a19aac7f651d8c2ded5466b2024f71c09 WatchSource:0}: Error finding container dce5e2de1885ccd8f1c972fd1c5ab39a19aac7f651d8c2ded5466b2024f71c09: Status 404 returned error can't find the container with id dce5e2de1885ccd8f1c972fd1c5ab39a19aac7f651d8c2ded5466b2024f71c09
Aug 03 16:59:21.682885 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.682795    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:21.710893 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.710855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq" event={"ID":"11ba03ed-acf9-4658-88bb-f880cfe3f129","Type":"ContainerStarted","Data":"f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f"}
Aug 03 16:59:21.711083 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.710908    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq" event={"ID":"11ba03ed-acf9-4658-88bb-f880cfe3f129","Type":"ContainerStarted","Data":"dce5e2de1885ccd8f1c972fd1c5ab39a19aac7f651d8c2ded5466b2024f71c09"}
Aug 03 16:59:21.711083 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.710938    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:21.712121 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.712098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" event={"ID":"500419f8-9c90-41f9-bb7b-fcdac23b752e","Type":"ContainerStarted","Data":"0ecc8565c3dfa535f8b1a57552a8514f48009a25eb27996c5a6550b8811636ed"}
Aug 03 16:59:21.732094 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:21.732041    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq" podStartSLOduration=1.732025196 podStartE2EDuration="1.732025196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:59:21.72924785 +0000 UTC m=+3321.278534167" watchObservedRunningTime="2026-08-03 16:59:21.732025196 +0000 UTC m=+3321.281311512"
Aug 03 16:59:22.718926 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:22.718885    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" event={"ID":"500419f8-9c90-41f9-bb7b-fcdac23b752e","Type":"ContainerStarted","Data":"4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6"}
Aug 03 16:59:22.719414 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:22.719125    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:22.737888 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:22.737836    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" podStartSLOduration=2.104988104 podStartE2EDuration="2.737817601s" totalImagesPullingTime="632.829497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:59:22.736892642 +0000 UTC m=+3322.286178955" watchObservedRunningTime="2026-08-03 16:59:22.737817601 +0000 UTC m=+3322.287103917"
Aug 03 16:59:28.731327 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:28.731293    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 16:59:32.721631 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:32.721596    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 16:59:44.189604 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:44.189565    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 16:59:51.166993 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:51.166955    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lstfn_8e84fafc-9d3d-4122-9b7c-67e622ad5028/payload-processing/0.log"
Aug 03 16:59:56.990129 ip-10-0-133-47 kubenswrapper[2560]: I0803 16:59:56.990092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:04.621739 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:04.621699    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"]
Aug 03 17:00:04.624194 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:04.624106    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29762895-v7h8l"]
Aug 03 17:00:04.987250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:04.987164    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12a021c2-2ae5-4784-94a8-709e17797fc8" path="/var/lib/kubelet/pods/12a021c2-2ae5-4784-94a8-709e17797fc8/volumes"
Aug 03 17:00:34.145662 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:34.145628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp_34eba44b-645f-48d4-8f6e-c9c1866401e0/payload-processing/0.log"
Aug 03 17:00:34.295933 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:34.295902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lstfn_8e84fafc-9d3d-4122-9b7c-67e622ad5028/payload-processing/0.log"
Aug 03 17:00:34.753296 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:34.753259    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:35.118395 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:35.118354    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:35.399328 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:35.399246    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:37.760163 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:37.760131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:37.861736 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:37.861703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:38.015549 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.015447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:38.116174 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.116133    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:38.958355 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.958322    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"]
Aug 03 17:00:38.962611 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.962588    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:38.965453 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.965433    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4df7e0-serving-cert\""
Aug 03 17:00:38.973294 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.973271    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"]
Aug 03 17:00:38.991522 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.991490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:38.991785 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.991593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shktq\" (UniqueName: \"kubernetes.io/projected/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-kube-api-access-shktq\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:38.991785 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.991736    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:38.991979 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:38.991955    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.092673 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.092630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.092673 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.092680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.092937 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:00:39.092813    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-serving-cert: secret "maas-api-e2e-ipp-cleanup-4df7e0-serving-cert" not found
Aug 03 17:00:39.092937 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.092876    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shktq\" (UniqueName: \"kubernetes.io/projected/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-kube-api-access-shktq\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.092937 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:00:39.092901    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls podName:62ed25e3-ff16-44ae-94a3-e2cdb942ea64 nodeName:}" failed. No retries permitted until 2026-08-03 17:00:39.592877962 +0000 UTC m=+3399.142164260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" (UID: "62ed25e3-ff16-44ae-94a3-e2cdb942ea64") : secret "maas-api-e2e-ipp-cleanup-4df7e0-serving-cert" not found
Aug 03 17:00:39.093408 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.093383    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.104754 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.104725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shktq\" (UniqueName: \"kubernetes.io/projected/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-kube-api-access-shktq\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.597600 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.597568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.600191 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.600155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.877795 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.877703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:39.928340 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:39.928305    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:40.030860 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:40.030826    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"]
Aug 03 17:00:40.033398 ip-10-0-133-47 kubenswrapper[2560]: W0803 17:00:40.033363    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ed25e3_ff16_44ae_94a3_e2cdb942ea64.slice/crio-cc041a11218f57b9cca1b8a11c751c3b86d97e613ad2a7c121f312028d7dbaf0 WatchSource:0}: Error finding container cc041a11218f57b9cca1b8a11c751c3b86d97e613ad2a7c121f312028d7dbaf0: Status 404 returned error can't find the container with id cc041a11218f57b9cca1b8a11c751c3b86d97e613ad2a7c121f312028d7dbaf0
Aug 03 17:00:40.062074 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:40.062041    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:00:40.130055 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:40.129967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" event={"ID":"62ed25e3-ff16-44ae-94a3-e2cdb942ea64","Type":"ContainerStarted","Data":"cc041a11218f57b9cca1b8a11c751c3b86d97e613ad2a7c121f312028d7dbaf0"}
Aug 03 17:00:41.137746 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:41.137711    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" event={"ID":"62ed25e3-ff16-44ae-94a3-e2cdb942ea64","Type":"ContainerStarted","Data":"75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe"}
Aug 03 17:00:41.138175 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:41.137888    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:00:41.159536 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:41.159481    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" podStartSLOduration=2.544289157 podStartE2EDuration="3.159465094s" totalImagesPullingTime="615.175937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:41.157423861 +0000 UTC m=+3400.706710175" watchObservedRunningTime="2026-08-03 17:00:41.159465094 +0000 UTC m=+3400.708751410"
Aug 03 17:00:47.148675 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:00:47.148648    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:01:01.567100 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:01.567019    2560 scope.go:117] "RemoveContainer" containerID="07a79442c934f7cdaf599a03c2cb3eb22e72b1e9c5fe078d9a1a8e97e681ed61"
Aug 03 17:01:02.913125 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:02.913091    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:03.568514 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:03.568481    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:03.635319 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:03.635279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:09.548143 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.548108    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"]
Aug 03 17:01:09.549139 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.549079    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" podUID="62ed25e3-ff16-44ae-94a3-e2cdb942ea64" containerName="maas-api" containerID="cri-o://75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe" gracePeriod=30
Aug 03 17:01:09.574588 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.574517    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:09.652150 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.652112    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:09.723186 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.723084    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:09.783995 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.783954    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:09.816597 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.816504    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:01:09.900636 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.900603    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls\") pod \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") "
Aug 03 17:01:09.900831 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.900674    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-openshift-service-ca\") pod \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") "
Aug 03 17:01:09.900831 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.900735    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shktq\" (UniqueName: \"kubernetes.io/projected/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-kube-api-access-shktq\") pod \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\" (UID: \"62ed25e3-ff16-44ae-94a3-e2cdb942ea64\") "
Aug 03 17:01:09.901081 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.901055    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62ed25e3-ff16-44ae-94a3-e2cdb942ea64" (UID: "62ed25e3-ff16-44ae-94a3-e2cdb942ea64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:09.902791 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.902759    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62ed25e3-ff16-44ae-94a3-e2cdb942ea64" (UID: "62ed25e3-ff16-44ae-94a3-e2cdb942ea64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 17:01:09.902966 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:09.902947    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-kube-api-access-shktq" (OuterVolumeSpecName: "kube-api-access-shktq") pod "62ed25e3-ff16-44ae-94a3-e2cdb942ea64" (UID: "62ed25e3-ff16-44ae-94a3-e2cdb942ea64"). InnerVolumeSpecName "kube-api-access-shktq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:10.001645 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.001591    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.001645 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.001640    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.001645 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.001650    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shktq\" (UniqueName: \"kubernetes.io/projected/62ed25e3-ff16-44ae-94a3-e2cdb942ea64-kube-api-access-shktq\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:10.198689 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.198601    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:10.291026 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.290988    2560 generic.go:356] "Generic (PLEG): container finished" podID="62ed25e3-ff16-44ae-94a3-e2cdb942ea64" containerID="75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe" exitCode=0
Aug 03 17:01:10.291198 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.291036    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"
Aug 03 17:01:10.291198 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.291058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" event={"ID":"62ed25e3-ff16-44ae-94a3-e2cdb942ea64","Type":"ContainerDied","Data":"75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe"}
Aug 03 17:01:10.291198 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.291109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p" event={"ID":"62ed25e3-ff16-44ae-94a3-e2cdb942ea64","Type":"ContainerDied","Data":"cc041a11218f57b9cca1b8a11c751c3b86d97e613ad2a7c121f312028d7dbaf0"}
Aug 03 17:01:10.291198 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.291140    2560 scope.go:117] "RemoveContainer" containerID="75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe"
Aug 03 17:01:10.301644 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.301622    2560 scope.go:117] "RemoveContainer" containerID="75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe"
Aug 03 17:01:10.301948 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:01:10.301927    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe\": container with ID starting with 75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe not found: ID does not exist" containerID="75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe"
Aug 03 17:01:10.302011 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.301958    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe"} err="failed to get container status \"75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe\": rpc error: code = NotFound desc = could not find container \"75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe\": container with ID starting with 75602ef1bbfeb9deae5151f4b403f6ad51571f71c51d0e167ce6a715162d91fe not found: ID does not exist"
Aug 03 17:01:10.317125 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.317092    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"]
Aug 03 17:01:10.323617 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.323586    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df7e0-67dcdc577-xlf9p"]
Aug 03 17:01:10.987716 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:10.987684    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62ed25e3-ff16-44ae-94a3-e2cdb942ea64" path="/var/lib/kubelet/pods/62ed25e3-ff16-44ae-94a3-e2cdb942ea64/volumes"
Aug 03 17:01:16.470333 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:16.470300    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:16.570446 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:16.570411    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:16.788627 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:16.788583    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:17.082201 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:17.082114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:36.619506 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.619462    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"]
Aug 03 17:01:36.620043 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.619727    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn" podUID="3125fdb7-e35d-40bd-8bad-2aa82a750015" containerName="maas-api" containerID="cri-o://76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e" gracePeriod=30
Aug 03 17:01:36.655710 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.655663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:36.684114 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.684082    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"]
Aug 03 17:01:36.684326 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.684296    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp" podUID="34eba44b-645f-48d4-8f6e-c9c1866401e0" containerName="payload-processing" containerID="cri-o://1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c" gracePeriod=30
Aug 03 17:01:36.687785 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.687712    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"]
Aug 03 17:01:36.687964 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.687940    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz" podUID="52452a70-45fd-42b2-9ae1-8ffdffaaab12" containerName="payload-pre-processing" containerID="cri-o://2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3" gracePeriod=30
Aug 03 17:01:36.723181 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.723121    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:36.819262 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.819222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:36.884577 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.884468    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:36.966201 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:36.966169    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 17:01:37.040682 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.040651    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 17:01:37.050189 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.050166    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 17:01:37.056631 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.056604    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3125fdb7-e35d-40bd-8bad-2aa82a750015-maas-api-tls\") pod \"3125fdb7-e35d-40bd-8bad-2aa82a750015\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") "
Aug 03 17:01:37.056793 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.056767    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3125fdb7-e35d-40bd-8bad-2aa82a750015-openshift-service-ca\") pod \"3125fdb7-e35d-40bd-8bad-2aa82a750015\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") "
Aug 03 17:01:37.057206 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.057058    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvnnk\" (UniqueName: \"kubernetes.io/projected/3125fdb7-e35d-40bd-8bad-2aa82a750015-kube-api-access-qvnnk\") pod \"3125fdb7-e35d-40bd-8bad-2aa82a750015\" (UID: \"3125fdb7-e35d-40bd-8bad-2aa82a750015\") "
Aug 03 17:01:37.057320 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.057216    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3125fdb7-e35d-40bd-8bad-2aa82a750015-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3125fdb7-e35d-40bd-8bad-2aa82a750015" (UID: "3125fdb7-e35d-40bd-8bad-2aa82a750015"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:37.057553 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.057506    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3125fdb7-e35d-40bd-8bad-2aa82a750015-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.059997 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.059960    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3125fdb7-e35d-40bd-8bad-2aa82a750015-kube-api-access-qvnnk" (OuterVolumeSpecName: "kube-api-access-qvnnk") pod "3125fdb7-e35d-40bd-8bad-2aa82a750015" (UID: "3125fdb7-e35d-40bd-8bad-2aa82a750015"). InnerVolumeSpecName "kube-api-access-qvnnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:37.060367 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.060347    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3125fdb7-e35d-40bd-8bad-2aa82a750015-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3125fdb7-e35d-40bd-8bad-2aa82a750015" (UID: "3125fdb7-e35d-40bd-8bad-2aa82a750015"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 17:01:37.158866 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.158778    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34eba44b-645f-48d4-8f6e-c9c1866401e0-plugins-config-volume\") pod \"34eba44b-645f-48d4-8f6e-c9c1866401e0\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") "
Aug 03 17:01:37.158866 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.158846    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52452a70-45fd-42b2-9ae1-8ffdffaaab12-plugins-config-volume\") pod \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") "
Aug 03 17:01:37.158866 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.158869    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfsql\" (UniqueName: \"kubernetes.io/projected/52452a70-45fd-42b2-9ae1-8ffdffaaab12-kube-api-access-zfsql\") pod \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\" (UID: \"52452a70-45fd-42b2-9ae1-8ffdffaaab12\") "
Aug 03 17:01:37.159146 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.158955    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tcm2\" (UniqueName: \"kubernetes.io/projected/34eba44b-645f-48d4-8f6e-c9c1866401e0-kube-api-access-4tcm2\") pod \"34eba44b-645f-48d4-8f6e-c9c1866401e0\" (UID: \"34eba44b-645f-48d4-8f6e-c9c1866401e0\") "
Aug 03 17:01:37.159146 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.159134    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3125fdb7-e35d-40bd-8bad-2aa82a750015-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.159146 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.159145    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvnnk\" (UniqueName: \"kubernetes.io/projected/3125fdb7-e35d-40bd-8bad-2aa82a750015-kube-api-access-qvnnk\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.159246 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.159203    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34eba44b-645f-48d4-8f6e-c9c1866401e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34eba44b-645f-48d4-8f6e-c9c1866401e0" (UID: "34eba44b-645f-48d4-8f6e-c9c1866401e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:37.159323 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.159291    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52452a70-45fd-42b2-9ae1-8ffdffaaab12-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52452a70-45fd-42b2-9ae1-8ffdffaaab12" (UID: "52452a70-45fd-42b2-9ae1-8ffdffaaab12"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:01:37.161141 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.161114    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52452a70-45fd-42b2-9ae1-8ffdffaaab12-kube-api-access-zfsql" (OuterVolumeSpecName: "kube-api-access-zfsql") pod "52452a70-45fd-42b2-9ae1-8ffdffaaab12" (UID: "52452a70-45fd-42b2-9ae1-8ffdffaaab12"). InnerVolumeSpecName "kube-api-access-zfsql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:37.161268 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.161221    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34eba44b-645f-48d4-8f6e-c9c1866401e0-kube-api-access-4tcm2" (OuterVolumeSpecName: "kube-api-access-4tcm2") pod "34eba44b-645f-48d4-8f6e-c9c1866401e0" (UID: "34eba44b-645f-48d4-8f6e-c9c1866401e0"). InnerVolumeSpecName "kube-api-access-4tcm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:01:37.260000 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.259959    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tcm2\" (UniqueName: \"kubernetes.io/projected/34eba44b-645f-48d4-8f6e-c9c1866401e0-kube-api-access-4tcm2\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.260000 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.259993    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34eba44b-645f-48d4-8f6e-c9c1866401e0-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.260000 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.260005    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52452a70-45fd-42b2-9ae1-8ffdffaaab12-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.260250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.260016    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfsql\" (UniqueName: \"kubernetes.io/projected/52452a70-45fd-42b2-9ae1-8ffdffaaab12-kube-api-access-zfsql\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:01:37.272647 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.272612    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:01:37.442043 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.441947    2560 generic.go:356] "Generic (PLEG): container finished" podID="3125fdb7-e35d-40bd-8bad-2aa82a750015" containerID="76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e" exitCode=0
Aug 03 17:01:37.442043 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.441992    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"
Aug 03 17:01:37.442043 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.442035    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn" event={"ID":"3125fdb7-e35d-40bd-8bad-2aa82a750015","Type":"ContainerDied","Data":"76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e"}
Aug 03 17:01:37.442323 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.442086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn" event={"ID":"3125fdb7-e35d-40bd-8bad-2aa82a750015","Type":"ContainerDied","Data":"f4d8b79c7183313935bf5ac0683a3c166278306cb66e0f7ea449f6d54e2e1c88"}
Aug 03 17:01:37.442323 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.442109    2560 scope.go:117] "RemoveContainer" containerID="76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e"
Aug 03 17:01:37.443541 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.443505    2560 generic.go:356] "Generic (PLEG): container finished" podID="52452a70-45fd-42b2-9ae1-8ffdffaaab12" containerID="2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3" exitCode=0
Aug 03 17:01:37.443638 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.443556    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"
Aug 03 17:01:37.443706 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.443651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz" event={"ID":"52452a70-45fd-42b2-9ae1-8ffdffaaab12","Type":"ContainerDied","Data":"2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3"}
Aug 03 17:01:37.443765 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.443708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz" event={"ID":"52452a70-45fd-42b2-9ae1-8ffdffaaab12","Type":"ContainerDied","Data":"3a56d489c9576e1cbb2f461f6f9f15d04840fe37752b9dc4841b88f6158662ff"}
Aug 03 17:01:37.445252 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.445181    2560 generic.go:356] "Generic (PLEG): container finished" podID="34eba44b-645f-48d4-8f6e-c9c1866401e0" containerID="1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c" exitCode=0
Aug 03 17:01:37.445252 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.445233    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"
Aug 03 17:01:37.445492 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.445238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp" event={"ID":"34eba44b-645f-48d4-8f6e-c9c1866401e0","Type":"ContainerDied","Data":"1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c"}
Aug 03 17:01:37.445492 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.445337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp" event={"ID":"34eba44b-645f-48d4-8f6e-c9c1866401e0","Type":"ContainerDied","Data":"d519fc9a55b2c9ee0af642e2e345939fea5174ecff4ff1daa9b450a2956972ae"}
Aug 03 17:01:37.452542 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.452495    2560 scope.go:117] "RemoveContainer" containerID="76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e"
Aug 03 17:01:37.453058 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:01:37.453030    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e\": container with ID starting with 76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e not found: ID does not exist" containerID="76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e"
Aug 03 17:01:37.453167 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.453066    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e"} err="failed to get container status \"76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e\": rpc error: code = NotFound desc = could not find container \"76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e\": container with ID starting with 76dd2ca052a1f44b935a0c536ce5fbc52f1ba93fc09079e701caf298aac4b02e not found: ID does not exist"
Aug 03 17:01:37.453167 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.453083    2560 scope.go:117] "RemoveContainer" containerID="2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3"
Aug 03 17:01:37.462579 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.462554    2560 scope.go:117] "RemoveContainer" containerID="2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3"
Aug 03 17:01:37.462897 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:01:37.462869    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3\": container with ID starting with 2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3 not found: ID does not exist" containerID="2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3"
Aug 03 17:01:37.462974 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.462909    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3"} err="failed to get container status \"2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3\": rpc error: code = NotFound desc = could not find container \"2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3\": container with ID starting with 2dbe0c126bca2c5a8485d5d14b45d6108cd9961d07c9a62bce7aa1ccbc630ae3 not found: ID does not exist"
Aug 03 17:01:37.462974 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.462928    2560 scope.go:117] "RemoveContainer" containerID="1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c"
Aug 03 17:01:37.468436 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.468410    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"]
Aug 03 17:01:37.472384 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.472362    2560 scope.go:117] "RemoveContainer" containerID="1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c"
Aug 03 17:01:37.472767 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:01:37.472744    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c\": container with ID starting with 1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c not found: ID does not exist" containerID="1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c"
Aug 03 17:01:37.472873 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.472773    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c"} err="failed to get container status \"1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c\": rpc error: code = NotFound desc = could not find container \"1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c\": container with ID starting with 1f02aa7338242b0b52d2b88e5dfa4017f9f8cd876c18c9864dab7e820789dc6c not found: ID does not exist"
Aug 03 17:01:37.474074 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.474052    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0e4b78-589fff968-zvttn"]
Aug 03 17:01:37.485245 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.485215    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"]
Aug 03 17:01:37.491661 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.491636    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0e4b78-796c5d99cb-m2njz"]
Aug 03 17:01:37.506434 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.503378    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"]
Aug 03 17:01:37.511988 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:37.511958    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0e4b78-7f87f65494-xv5vp"]
Aug 03 17:01:38.987163 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:38.987127    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3125fdb7-e35d-40bd-8bad-2aa82a750015" path="/var/lib/kubelet/pods/3125fdb7-e35d-40bd-8bad-2aa82a750015/volumes"
Aug 03 17:01:38.987565 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:38.987488    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34eba44b-645f-48d4-8f6e-c9c1866401e0" path="/var/lib/kubelet/pods/34eba44b-645f-48d4-8f6e-c9c1866401e0/volumes"
Aug 03 17:01:38.987826 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:38.987812    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52452a70-45fd-42b2-9ae1-8ffdffaaab12" path="/var/lib/kubelet/pods/52452a70-45fd-42b2-9ae1-8ffdffaaab12/volumes"
Aug 03 17:01:42.425390 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:01:42.425356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:19.374319 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:19.374275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:20.134631 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:20.134593    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:26.447422 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:26.447335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:41.083551 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:41.083493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:41.184095 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:41.184063    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:41.434828 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:41.434738    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:41.742860 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:41.742768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:52.298219 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.298179    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"]
Aug 03 17:02:52.298888 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.298470    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" podUID="500419f8-9c90-41f9-bb7b-fcdac23b752e" containerName="maas-api" containerID="cri-o://4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6" gracePeriod=30
Aug 03 17:02:52.336486 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.336455    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:52.386867 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.386829    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"]
Aug 03 17:02:52.387165 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.387106    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq" podUID="11ba03ed-acf9-4658-88bb-f880cfe3f129" containerName="payload-pre-processing" containerID="cri-o://f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f" gracePeriod=30
Aug 03 17:02:52.433105 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.433072    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:52.502121 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.502088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:52.566467 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.566430    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:52.648062 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.648025    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 17:02:52.684691 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.684663    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 17:02:52.767784 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.767753    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls\") pod \"500419f8-9c90-41f9-bb7b-fcdac23b752e\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") "
Aug 03 17:02:52.767970 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.767797    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcgg4\" (UniqueName: \"kubernetes.io/projected/500419f8-9c90-41f9-bb7b-fcdac23b752e-kube-api-access-mcgg4\") pod \"500419f8-9c90-41f9-bb7b-fcdac23b752e\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") "
Aug 03 17:02:52.767970 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.767831    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/500419f8-9c90-41f9-bb7b-fcdac23b752e-openshift-service-ca\") pod \"500419f8-9c90-41f9-bb7b-fcdac23b752e\" (UID: \"500419f8-9c90-41f9-bb7b-fcdac23b752e\") "
Aug 03 17:02:52.767970 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.767859    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqd49\" (UniqueName: \"kubernetes.io/projected/11ba03ed-acf9-4658-88bb-f880cfe3f129-kube-api-access-kqd49\") pod \"11ba03ed-acf9-4658-88bb-f880cfe3f129\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") "
Aug 03 17:02:52.767970 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.767930    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ba03ed-acf9-4658-88bb-f880cfe3f129-plugins-config-volume\") pod \"11ba03ed-acf9-4658-88bb-f880cfe3f129\" (UID: \"11ba03ed-acf9-4658-88bb-f880cfe3f129\") "
Aug 03 17:02:52.768285 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.768257    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/500419f8-9c90-41f9-bb7b-fcdac23b752e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "500419f8-9c90-41f9-bb7b-fcdac23b752e" (UID: "500419f8-9c90-41f9-bb7b-fcdac23b752e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:02:52.768359 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.768333    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ba03ed-acf9-4658-88bb-f880cfe3f129-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11ba03ed-acf9-4658-88bb-f880cfe3f129" (UID: "11ba03ed-acf9-4658-88bb-f880cfe3f129"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:02:52.770030 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.770008    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "500419f8-9c90-41f9-bb7b-fcdac23b752e" (UID: "500419f8-9c90-41f9-bb7b-fcdac23b752e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 17:02:52.770125 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.770060    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ba03ed-acf9-4658-88bb-f880cfe3f129-kube-api-access-kqd49" (OuterVolumeSpecName: "kube-api-access-kqd49") pod "11ba03ed-acf9-4658-88bb-f880cfe3f129" (UID: "11ba03ed-acf9-4658-88bb-f880cfe3f129"). InnerVolumeSpecName "kube-api-access-kqd49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:02:52.770125 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.770093    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/500419f8-9c90-41f9-bb7b-fcdac23b752e-kube-api-access-mcgg4" (OuterVolumeSpecName: "kube-api-access-mcgg4") pod "500419f8-9c90-41f9-bb7b-fcdac23b752e" (UID: "500419f8-9c90-41f9-bb7b-fcdac23b752e"). InnerVolumeSpecName "kube-api-access-mcgg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:02:52.847395 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.847315    2560 generic.go:356] "Generic (PLEG): container finished" podID="500419f8-9c90-41f9-bb7b-fcdac23b752e" containerID="4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6" exitCode=0
Aug 03 17:02:52.847395 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.847365    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"
Aug 03 17:02:52.847642 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.847409    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" event={"ID":"500419f8-9c90-41f9-bb7b-fcdac23b752e","Type":"ContainerDied","Data":"4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6"}
Aug 03 17:02:52.847642 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.847464    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc" event={"ID":"500419f8-9c90-41f9-bb7b-fcdac23b752e","Type":"ContainerDied","Data":"0ecc8565c3dfa535f8b1a57552a8514f48009a25eb27996c5a6550b8811636ed"}
Aug 03 17:02:52.847642 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.847491    2560 scope.go:117] "RemoveContainer" containerID="4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6"
Aug 03 17:02:52.848788 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.848763    2560 generic.go:356] "Generic (PLEG): container finished" podID="11ba03ed-acf9-4658-88bb-f880cfe3f129" containerID="f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f" exitCode=0
Aug 03 17:02:52.848883 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.848793    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq" event={"ID":"11ba03ed-acf9-4658-88bb-f880cfe3f129","Type":"ContainerDied","Data":"f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f"}
Aug 03 17:02:52.848883 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.848825    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"
Aug 03 17:02:52.848883 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.848834    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq" event={"ID":"11ba03ed-acf9-4658-88bb-f880cfe3f129","Type":"ContainerDied","Data":"dce5e2de1885ccd8f1c972fd1c5ab39a19aac7f651d8c2ded5466b2024f71c09"}
Aug 03 17:02:52.858323 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.858309    2560 scope.go:117] "RemoveContainer" containerID="4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6"
Aug 03 17:02:52.858618 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:02:52.858596    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6\": container with ID starting with 4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6 not found: ID does not exist" containerID="4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6"
Aug 03 17:02:52.858692 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.858633    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6"} err="failed to get container status \"4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6\": rpc error: code = NotFound desc = could not find container \"4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6\": container with ID starting with 4e1ddf3019bc2bc9833113aac16970a746bf9324aae73361d5645780ebc915d6 not found: ID does not exist"
Aug 03 17:02:52.858692 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.858656    2560 scope.go:117] "RemoveContainer" containerID="f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f"
Aug 03 17:02:52.866981 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.866963    2560 scope.go:117] "RemoveContainer" containerID="f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f"
Aug 03 17:02:52.867248 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:02:52.867229    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f\": container with ID starting with f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f not found: ID does not exist" containerID="f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f"
Aug 03 17:02:52.867289 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.867257    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f"} err="failed to get container status \"f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f\": rpc error: code = NotFound desc = could not find container \"f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f\": container with ID starting with f27e061629222d856e6ceb7199c84f368ea5315f6da377e1b23a5e69b46b459f not found: ID does not exist"
Aug 03 17:02:52.868714 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.868695    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11ba03ed-acf9-4658-88bb-f880cfe3f129-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:02:52.868764 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.868718    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/500419f8-9c90-41f9-bb7b-fcdac23b752e-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:02:52.868764 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.868727    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcgg4\" (UniqueName: \"kubernetes.io/projected/500419f8-9c90-41f9-bb7b-fcdac23b752e-kube-api-access-mcgg4\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:02:52.868764 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.868735    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/500419f8-9c90-41f9-bb7b-fcdac23b752e-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:02:52.868764 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.868744    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqd49\" (UniqueName: \"kubernetes.io/projected/11ba03ed-acf9-4658-88bb-f880cfe3f129-kube-api-access-kqd49\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:02:52.872500 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.872480    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"]
Aug 03 17:02:52.876426 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.876406    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cf8ed9-5bd87c87d8-f69pc"]
Aug 03 17:02:52.886889 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.886860    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"]
Aug 03 17:02:52.891291 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.891269    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cf8ed9-5f4dd6557b-ggwhq"]
Aug 03 17:02:52.987511 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.987476    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11ba03ed-acf9-4658-88bb-f880cfe3f129" path="/var/lib/kubelet/pods/11ba03ed-acf9-4658-88bb-f880cfe3f129/volumes"
Aug 03 17:02:52.987937 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:52.987921    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="500419f8-9c90-41f9-bb7b-fcdac23b752e" path="/var/lib/kubelet/pods/500419f8-9c90-41f9-bb7b-fcdac23b752e/volumes"
Aug 03 17:02:53.035136 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:53.035101    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:59.243945 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:59.243911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:59.357927 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:59.357890    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:59.585657 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:59.585625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:02:59.873211 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:02:59.873106    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:10.608871 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.608838    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"]
Aug 03 17:03:10.609356 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.609135    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" podUID="f4d52db1-d273-40b1-9793-d4af1b53e77e" containerName="maas-api" containerID="cri-o://2f796466664325cbc214933f6946b093120421c86f21d92ae1e72ec73a71b5b3" gracePeriod=30
Aug 03 17:03:10.640015 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.639980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:10.672040 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.671997    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"]
Aug 03 17:03:10.672613 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.672232    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5" podUID="c61a37f6-12e6-410a-8f01-74e7aecb5f39" containerName="payload-processing" containerID="cri-o://c85ea1258163590c74856abdf3901bfe1698806df588cd58f0e0f56665420a1f" gracePeriod=30
Aug 03 17:03:10.679833 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.679809    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"]
Aug 03 17:03:10.680064 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.680042    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8" podUID="736ecbd0-5e7f-4e7b-b951-632c189706fd" containerName="payload-pre-processing" containerID="cri-o://a81235888fdef6538b5a7f3e13ca04d4d7b6cca708e7cb4467a2b2e17590cea6" gracePeriod=30
Aug 03 17:03:10.711827 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.711150    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:10.774245 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.774207    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:10.875423 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.875322    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:10.955515 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.955485    2560 generic.go:356] "Generic (PLEG): container finished" podID="c61a37f6-12e6-410a-8f01-74e7aecb5f39" containerID="c85ea1258163590c74856abdf3901bfe1698806df588cd58f0e0f56665420a1f" exitCode=0
Aug 03 17:03:10.955795 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.955568    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5" event={"ID":"c61a37f6-12e6-410a-8f01-74e7aecb5f39","Type":"ContainerDied","Data":"c85ea1258163590c74856abdf3901bfe1698806df588cd58f0e0f56665420a1f"}
Aug 03 17:03:10.955795 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.955631    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5" event={"ID":"c61a37f6-12e6-410a-8f01-74e7aecb5f39","Type":"ContainerDied","Data":"871083835d26f7cee974382570d93edeb0452370f482917f8de972330e431af5"}
Aug 03 17:03:10.955795 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.955654    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="871083835d26f7cee974382570d93edeb0452370f482917f8de972330e431af5"
Aug 03 17:03:10.957279 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.957247    2560 generic.go:356] "Generic (PLEG): container finished" podID="f4d52db1-d273-40b1-9793-d4af1b53e77e" containerID="2f796466664325cbc214933f6946b093120421c86f21d92ae1e72ec73a71b5b3" exitCode=0
Aug 03 17:03:10.957418 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.957323    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" event={"ID":"f4d52db1-d273-40b1-9793-d4af1b53e77e","Type":"ContainerDied","Data":"2f796466664325cbc214933f6946b093120421c86f21d92ae1e72ec73a71b5b3"}
Aug 03 17:03:10.958918 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.958896    2560 generic.go:356] "Generic (PLEG): container finished" podID="736ecbd0-5e7f-4e7b-b951-632c189706fd" containerID="a81235888fdef6538b5a7f3e13ca04d4d7b6cca708e7cb4467a2b2e17590cea6" exitCode=0
Aug 03 17:03:10.959009 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.958979    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8" event={"ID":"736ecbd0-5e7f-4e7b-b951-632c189706fd","Type":"ContainerDied","Data":"a81235888fdef6538b5a7f3e13ca04d4d7b6cca708e7cb4467a2b2e17590cea6"}
Aug 03 17:03:10.970411 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.970386    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 17:03:10.982107 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.982083    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 17:03:10.987986 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:10.987873    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 17:03:11.141992 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.141883    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwb5l\" (UniqueName: \"kubernetes.io/projected/c61a37f6-12e6-410a-8f01-74e7aecb5f39-kube-api-access-rwb5l\") pod \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") "
Aug 03 17:03:11.141992 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.141944    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/736ecbd0-5e7f-4e7b-b951-632c189706fd-plugins-config-volume\") pod \"736ecbd0-5e7f-4e7b-b951-632c189706fd\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") "
Aug 03 17:03:11.142250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142005    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whr67\" (UniqueName: \"kubernetes.io/projected/f4d52db1-d273-40b1-9793-d4af1b53e77e-kube-api-access-whr67\") pod \"f4d52db1-d273-40b1-9793-d4af1b53e77e\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") "
Aug 03 17:03:11.142250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142046    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls\") pod \"f4d52db1-d273-40b1-9793-d4af1b53e77e\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") "
Aug 03 17:03:11.142250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142153    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4d52db1-d273-40b1-9793-d4af1b53e77e-openshift-service-ca\") pod \"f4d52db1-d273-40b1-9793-d4af1b53e77e\" (UID: \"f4d52db1-d273-40b1-9793-d4af1b53e77e\") "
Aug 03 17:03:11.142250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142200    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61a37f6-12e6-410a-8f01-74e7aecb5f39-plugins-config-volume\") pod \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\" (UID: \"c61a37f6-12e6-410a-8f01-74e7aecb5f39\") "
Aug 03 17:03:11.142250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142248    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzsl2\" (UniqueName: \"kubernetes.io/projected/736ecbd0-5e7f-4e7b-b951-632c189706fd-kube-api-access-zzsl2\") pod \"736ecbd0-5e7f-4e7b-b951-632c189706fd\" (UID: \"736ecbd0-5e7f-4e7b-b951-632c189706fd\") "
Aug 03 17:03:11.142573 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142385    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/736ecbd0-5e7f-4e7b-b951-632c189706fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "736ecbd0-5e7f-4e7b-b951-632c189706fd" (UID: "736ecbd0-5e7f-4e7b-b951-632c189706fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:11.142573 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142511    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d52db1-d273-40b1-9793-d4af1b53e77e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4d52db1-d273-40b1-9793-d4af1b53e77e" (UID: "f4d52db1-d273-40b1-9793-d4af1b53e77e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:11.142773 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142752    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/736ecbd0-5e7f-4e7b-b951-632c189706fd-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.142839 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142779    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4d52db1-d273-40b1-9793-d4af1b53e77e-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.142921 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.142893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c61a37f6-12e6-410a-8f01-74e7aecb5f39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c61a37f6-12e6-410a-8f01-74e7aecb5f39" (UID: "c61a37f6-12e6-410a-8f01-74e7aecb5f39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:11.144307 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.144276    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d52db1-d273-40b1-9793-d4af1b53e77e-kube-api-access-whr67" (OuterVolumeSpecName: "kube-api-access-whr67") pod "f4d52db1-d273-40b1-9793-d4af1b53e77e" (UID: "f4d52db1-d273-40b1-9793-d4af1b53e77e"). InnerVolumeSpecName "kube-api-access-whr67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:03:11.144410 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.144305    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4d52db1-d273-40b1-9793-d4af1b53e77e" (UID: "f4d52db1-d273-40b1-9793-d4af1b53e77e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 17:03:11.144410 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.144313    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c61a37f6-12e6-410a-8f01-74e7aecb5f39-kube-api-access-rwb5l" (OuterVolumeSpecName: "kube-api-access-rwb5l") pod "c61a37f6-12e6-410a-8f01-74e7aecb5f39" (UID: "c61a37f6-12e6-410a-8f01-74e7aecb5f39"). InnerVolumeSpecName "kube-api-access-rwb5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:03:11.144517 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.144471    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/736ecbd0-5e7f-4e7b-b951-632c189706fd-kube-api-access-zzsl2" (OuterVolumeSpecName: "kube-api-access-zzsl2") pod "736ecbd0-5e7f-4e7b-b951-632c189706fd" (UID: "736ecbd0-5e7f-4e7b-b951-632c189706fd"). InnerVolumeSpecName "kube-api-access-zzsl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:03:11.243387 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.243350    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4d52db1-d273-40b1-9793-d4af1b53e77e-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.243387 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.243380    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c61a37f6-12e6-410a-8f01-74e7aecb5f39-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.243387 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.243390    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzsl2\" (UniqueName: \"kubernetes.io/projected/736ecbd0-5e7f-4e7b-b951-632c189706fd-kube-api-access-zzsl2\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.243668 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.243400    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwb5l\" (UniqueName: \"kubernetes.io/projected/c61a37f6-12e6-410a-8f01-74e7aecb5f39-kube-api-access-rwb5l\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.243668 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.243409    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whr67\" (UniqueName: \"kubernetes.io/projected/f4d52db1-d273-40b1-9793-d4af1b53e77e-kube-api-access-whr67\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:11.256096 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.256068    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:11.965693 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.965605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54" event={"ID":"f4d52db1-d273-40b1-9793-d4af1b53e77e","Type":"ContainerDied","Data":"e129fffa955ce4c5d355505f06c899bbfe8f40159541e6bc3d842f9e6b2adad1"}
Aug 03 17:03:11.965693 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.965636    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"
Aug 03 17:03:11.965693 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.965680    2560 scope.go:117] "RemoveContainer" containerID="2f796466664325cbc214933f6946b093120421c86f21d92ae1e72ec73a71b5b3"
Aug 03 17:03:11.967050 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.967010    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"
Aug 03 17:03:11.967115 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.967053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8" event={"ID":"736ecbd0-5e7f-4e7b-b951-632c189706fd","Type":"ContainerDied","Data":"d01f92d7b69b9edf9a03a2817aa5c49e14a73b30ad1e6e0fd0fedd8a83a74933"}
Aug 03 17:03:11.967228 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.967209    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"
Aug 03 17:03:11.976346 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.976163    2560 scope.go:117] "RemoveContainer" containerID="a81235888fdef6538b5a7f3e13ca04d4d7b6cca708e7cb4467a2b2e17590cea6"
Aug 03 17:03:11.995036 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:11.995009    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"]
Aug 03 17:03:12.002565 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.002523    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0fae8e-f7d89477b-g6xg8"]
Aug 03 17:03:12.012914 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.012887    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"]
Aug 03 17:03:12.016747 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.016726    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0fae8e-5bf68f678c-dgn54"]
Aug 03 17:03:12.027250 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.027222    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"]
Aug 03 17:03:12.031229 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.031204    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0fae8e-8bd69d798-bnpd5"]
Aug 03 17:03:12.986940 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.986907    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="736ecbd0-5e7f-4e7b-b951-632c189706fd" path="/var/lib/kubelet/pods/736ecbd0-5e7f-4e7b-b951-632c189706fd/volumes"
Aug 03 17:03:12.987325 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.987267    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c61a37f6-12e6-410a-8f01-74e7aecb5f39" path="/var/lib/kubelet/pods/c61a37f6-12e6-410a-8f01-74e7aecb5f39/volumes"
Aug 03 17:03:12.987635 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:12.987621    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d52db1-d273-40b1-9793-d4af1b53e77e" path="/var/lib/kubelet/pods/f4d52db1-d273-40b1-9793-d4af1b53e77e/volumes"
Aug 03 17:03:17.850464 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:17.850424    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:17.950903 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:17.950847    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:18.182299 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.182194    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:18.460425 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.460336    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:18.588704 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.588671    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"]
Aug 03 17:03:18.589132 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589114    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="736ecbd0-5e7f-4e7b-b951-632c189706fd" containerName="payload-pre-processing"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589135    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="736ecbd0-5e7f-4e7b-b951-632c189706fd" containerName="payload-pre-processing"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589152    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52452a70-45fd-42b2-9ae1-8ffdffaaab12" containerName="payload-pre-processing"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589160    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="52452a70-45fd-42b2-9ae1-8ffdffaaab12" containerName="payload-pre-processing"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589178    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d52db1-d273-40b1-9793-d4af1b53e77e" containerName="maas-api"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589186    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d52db1-d273-40b1-9793-d4af1b53e77e" containerName="maas-api"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589200    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ba03ed-acf9-4658-88bb-f880cfe3f129" containerName="payload-pre-processing"
Aug 03 17:03:18.589221 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589208    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ba03ed-acf9-4658-88bb-f880cfe3f129" containerName="payload-pre-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589226    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3125fdb7-e35d-40bd-8bad-2aa82a750015" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589235    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3125fdb7-e35d-40bd-8bad-2aa82a750015" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589260    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c61a37f6-12e6-410a-8f01-74e7aecb5f39" containerName="payload-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589269    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c61a37f6-12e6-410a-8f01-74e7aecb5f39" containerName="payload-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589279    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34eba44b-645f-48d4-8f6e-c9c1866401e0" containerName="payload-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589289    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="34eba44b-645f-48d4-8f6e-c9c1866401e0" containerName="payload-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589299    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62ed25e3-ff16-44ae-94a3-e2cdb942ea64" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589307    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="62ed25e3-ff16-44ae-94a3-e2cdb942ea64" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589317    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500419f8-9c90-41f9-bb7b-fcdac23b752e" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589326    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="500419f8-9c90-41f9-bb7b-fcdac23b752e" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589421    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="11ba03ed-acf9-4658-88bb-f880cfe3f129" containerName="payload-pre-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589434    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="62ed25e3-ff16-44ae-94a3-e2cdb942ea64" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589445    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3125fdb7-e35d-40bd-8bad-2aa82a750015" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589456    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="736ecbd0-5e7f-4e7b-b951-632c189706fd" containerName="payload-pre-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589466    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c61a37f6-12e6-410a-8f01-74e7aecb5f39" containerName="payload-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589475    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="500419f8-9c90-41f9-bb7b-fcdac23b752e" containerName="maas-api"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589486    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="52452a70-45fd-42b2-9ae1-8ffdffaaab12" containerName="payload-pre-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589496    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="34eba44b-645f-48d4-8f6e-c9c1866401e0" containerName="payload-processing"
Aug 03 17:03:18.589568 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.589506    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d52db1-d273-40b1-9793-d4af1b53e77e" containerName="maas-api"
Aug 03 17:03:18.604646 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.604503    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"]
Aug 03 17:03:18.605940 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.605019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:18.609145 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.609123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-98rrw\""
Aug 03 17:03:18.610148 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.610124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4\" (UID: \"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:18.711229 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.711134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4\" (UID: \"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:18.711869 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.711845    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4\" (UID: \"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:18.920761 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:18.920724    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:19.062107 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:19.062083    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"]
Aug 03 17:03:19.064300 ip-10-0-133-47 kubenswrapper[2560]: W0803 17:03:19.064266    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c29b03_f7a5_4fe5_9be7_52f28d7dec2b.slice/crio-6623ca41fe08fe3ab32349710be76e8fa4edb0af9a8b9cf587d47cf243a14962 WatchSource:0}: Error finding container 6623ca41fe08fe3ab32349710be76e8fa4edb0af9a8b9cf587d47cf243a14962: Status 404 returned error can't find the container with id 6623ca41fe08fe3ab32349710be76e8fa4edb0af9a8b9cf587d47cf243a14962
Aug 03 17:03:20.014883 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:20.014851    2560 generic.go:356] "Generic (PLEG): container finished" podID="79c29b03-f7a5-4fe5-9be7-52f28d7dec2b" containerID="660a9e3277dddc509a3dc152bc5c6fa053af08974d4eb67ccabb5162f2977257" exitCode=0
Aug 03 17:03:20.015247 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:20.014897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4" event={"ID":"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b","Type":"ContainerDied","Data":"660a9e3277dddc509a3dc152bc5c6fa053af08974d4eb67ccabb5162f2977257"}
Aug 03 17:03:20.015247 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:20.014966    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4" event={"ID":"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b","Type":"ContainerStarted","Data":"6623ca41fe08fe3ab32349710be76e8fa4edb0af9a8b9cf587d47cf243a14962"}
Aug 03 17:03:21.158780 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:21.158750    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:21.241786 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:21.241753    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b-maas-api-service-ca\") pod \"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b\" (UID: \"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b\") "
Aug 03 17:03:21.242109 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:21.242086    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "79c29b03-f7a5-4fe5-9be7-52f28d7dec2b" (UID: "79c29b03-f7a5-4fe5-9be7-52f28d7dec2b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:21.343387 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:21.343354    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b-maas-api-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:22.026917 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:22.026886    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"
Aug 03 17:03:22.027077 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:22.026931    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4" event={"ID":"79c29b03-f7a5-4fe5-9be7-52f28d7dec2b","Type":"ContainerDied","Data":"6623ca41fe08fe3ab32349710be76e8fa4edb0af9a8b9cf587d47cf243a14962"}
Aug 03 17:03:22.027077 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:22.026975    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6623ca41fe08fe3ab32349710be76e8fa4edb0af9a8b9cf587d47cf243a14962"
Aug 03 17:03:29.002254 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.002208    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"]
Aug 03 17:03:29.002902 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.002456    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" podUID="9df64002-fab9-41da-8eb5-033d08ac228e" containerName="maas-api" containerID="cri-o://0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9" gracePeriod=30
Aug 03 17:03:29.038279 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.038231    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:29.059350 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.059315    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"]
Aug 03 17:03:29.059628 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.059587    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm" podUID="0118505a-4d26-4cfd-b95a-32e5dbd3927b" containerName="payload-processing" containerID="cri-o://e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7" gracePeriod=30
Aug 03 17:03:29.070789 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.070719    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"]
Aug 03 17:03:29.071009 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.070958    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99" podUID="6d894c9e-ca69-4931-8184-a792d69d2e6c" containerName="payload-pre-processing" containerID="cri-o://8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52" gracePeriod=30
Aug 03 17:03:29.199460 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.199396    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"]
Aug 03 17:03:29.202543 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.202496    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7055f0-nzhl4"]
Aug 03 17:03:29.364210 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.364185    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 17:03:29.367236 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.367215    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 17:03:29.381458 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.381434    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 17:03:29.424406 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424377    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d894c9e-ca69-4931-8184-a792d69d2e6c-plugins-config-volume\") pod \"6d894c9e-ca69-4931-8184-a792d69d2e6c\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") "
Aug 03 17:03:29.424607 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424414    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlvlv\" (UniqueName: \"kubernetes.io/projected/9df64002-fab9-41da-8eb5-033d08ac228e-kube-api-access-jlvlv\") pod \"9df64002-fab9-41da-8eb5-033d08ac228e\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") "
Aug 03 17:03:29.424607 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424448    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df64002-fab9-41da-8eb5-033d08ac228e-openshift-service-ca\") pod \"9df64002-fab9-41da-8eb5-033d08ac228e\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") "
Aug 03 17:03:29.424607 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424473    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0118505a-4d26-4cfd-b95a-32e5dbd3927b-plugins-config-volume\") pod \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") "
Aug 03 17:03:29.424607 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424541    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls\") pod \"9df64002-fab9-41da-8eb5-033d08ac228e\" (UID: \"9df64002-fab9-41da-8eb5-033d08ac228e\") "
Aug 03 17:03:29.424607 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424571    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf9j7\" (UniqueName: \"kubernetes.io/projected/0118505a-4d26-4cfd-b95a-32e5dbd3927b-kube-api-access-bf9j7\") pod \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\" (UID: \"0118505a-4d26-4cfd-b95a-32e5dbd3927b\") "
Aug 03 17:03:29.424898 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424618    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnfxg\" (UniqueName: \"kubernetes.io/projected/6d894c9e-ca69-4931-8184-a792d69d2e6c-kube-api-access-rnfxg\") pod \"6d894c9e-ca69-4931-8184-a792d69d2e6c\" (UID: \"6d894c9e-ca69-4931-8184-a792d69d2e6c\") "
Aug 03 17:03:29.424898 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424835    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9df64002-fab9-41da-8eb5-033d08ac228e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9df64002-fab9-41da-8eb5-033d08ac228e" (UID: "9df64002-fab9-41da-8eb5-033d08ac228e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:29.424898 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424848    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d894c9e-ca69-4931-8184-a792d69d2e6c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d894c9e-ca69-4931-8184-a792d69d2e6c" (UID: "6d894c9e-ca69-4931-8184-a792d69d2e6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:29.425065 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424938    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d894c9e-ca69-4931-8184-a792d69d2e6c-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:29.425065 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.424957    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df64002-fab9-41da-8eb5-033d08ac228e-openshift-service-ca\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:29.425226 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.425197    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0118505a-4d26-4cfd-b95a-32e5dbd3927b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0118505a-4d26-4cfd-b95a-32e5dbd3927b" (UID: "0118505a-4d26-4cfd-b95a-32e5dbd3927b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 03 17:03:29.426839 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.426806    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9df64002-fab9-41da-8eb5-033d08ac228e" (UID: "9df64002-fab9-41da-8eb5-033d08ac228e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 03 17:03:29.426936 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.426913    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d894c9e-ca69-4931-8184-a792d69d2e6c-kube-api-access-rnfxg" (OuterVolumeSpecName: "kube-api-access-rnfxg") pod "6d894c9e-ca69-4931-8184-a792d69d2e6c" (UID: "6d894c9e-ca69-4931-8184-a792d69d2e6c"). InnerVolumeSpecName "kube-api-access-rnfxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:03:29.427074 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.427052    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0118505a-4d26-4cfd-b95a-32e5dbd3927b-kube-api-access-bf9j7" (OuterVolumeSpecName: "kube-api-access-bf9j7") pod "0118505a-4d26-4cfd-b95a-32e5dbd3927b" (UID: "0118505a-4d26-4cfd-b95a-32e5dbd3927b"). InnerVolumeSpecName "kube-api-access-bf9j7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:03:29.427473 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.427099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df64002-fab9-41da-8eb5-033d08ac228e-kube-api-access-jlvlv" (OuterVolumeSpecName: "kube-api-access-jlvlv") pod "9df64002-fab9-41da-8eb5-033d08ac228e" (UID: "9df64002-fab9-41da-8eb5-033d08ac228e"). InnerVolumeSpecName "kube-api-access-jlvlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 03 17:03:29.526148 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.526115    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0118505a-4d26-4cfd-b95a-32e5dbd3927b-plugins-config-volume\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:29.526148 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.526144    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df64002-fab9-41da-8eb5-033d08ac228e-maas-api-tls\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:29.526148 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.526154    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf9j7\" (UniqueName: \"kubernetes.io/projected/0118505a-4d26-4cfd-b95a-32e5dbd3927b-kube-api-access-bf9j7\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:29.526439 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.526163    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnfxg\" (UniqueName: \"kubernetes.io/projected/6d894c9e-ca69-4931-8184-a792d69d2e6c-kube-api-access-rnfxg\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:29.526439 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:29.526173    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlvlv\" (UniqueName: \"kubernetes.io/projected/9df64002-fab9-41da-8eb5-033d08ac228e-kube-api-access-jlvlv\") on node \"ip-10-0-133-47.ec2.internal\" DevicePath \"\""
Aug 03 17:03:30.077121 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.077090    2560 generic.go:356] "Generic (PLEG): container finished" podID="0118505a-4d26-4cfd-b95a-32e5dbd3927b" containerID="e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7" exitCode=0
Aug 03 17:03:30.077570 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.077133    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"
Aug 03 17:03:30.077570 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.077173    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm" event={"ID":"0118505a-4d26-4cfd-b95a-32e5dbd3927b","Type":"ContainerDied","Data":"e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7"}
Aug 03 17:03:30.077570 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.077224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm" event={"ID":"0118505a-4d26-4cfd-b95a-32e5dbd3927b","Type":"ContainerDied","Data":"0eb1c7c25da05b593f57ecdc521b0e1878d96d12287dee8d660daeff669ac2ae"}
Aug 03 17:03:30.077570 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.077245    2560 scope.go:117] "RemoveContainer" containerID="e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7"
Aug 03 17:03:30.078550 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.078498    2560 generic.go:356] "Generic (PLEG): container finished" podID="6d894c9e-ca69-4931-8184-a792d69d2e6c" containerID="8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52" exitCode=0
Aug 03 17:03:30.078622 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.078556    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"
Aug 03 17:03:30.078622 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.078566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99" event={"ID":"6d894c9e-ca69-4931-8184-a792d69d2e6c","Type":"ContainerDied","Data":"8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52"}
Aug 03 17:03:30.078622 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.078609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99" event={"ID":"6d894c9e-ca69-4931-8184-a792d69d2e6c","Type":"ContainerDied","Data":"582638b486342cfd091e3318191f9c326c319c41be7c859be604203dd2db20da"}
Aug 03 17:03:30.080099 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.080074    2560 generic.go:356] "Generic (PLEG): container finished" podID="9df64002-fab9-41da-8eb5-033d08ac228e" containerID="0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9" exitCode=0
Aug 03 17:03:30.080231 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.080126    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"
Aug 03 17:03:30.080231 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.080132    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" event={"ID":"9df64002-fab9-41da-8eb5-033d08ac228e","Type":"ContainerDied","Data":"0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9"}
Aug 03 17:03:30.080231 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.080223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7" event={"ID":"9df64002-fab9-41da-8eb5-033d08ac228e","Type":"ContainerDied","Data":"0d1a74234ecb484b917a7375405eac5cdeb5bb481304333640d69899ad3f630e"}
Aug 03 17:03:30.087034 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.087016    2560 scope.go:117] "RemoveContainer" containerID="e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7"
Aug 03 17:03:30.087341 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:03:30.087315    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7\": container with ID starting with e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7 not found: ID does not exist" containerID="e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7"
Aug 03 17:03:30.087469 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.087353    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7"} err="failed to get container status \"e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7\": rpc error: code = NotFound desc = could not find container \"e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7\": container with ID starting with e7e203f637eb475b0769dc19faf02caad96893a2be7b8f3568e543cf1ee705f7 not found: ID does not exist"
Aug 03 17:03:30.087469 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.087375    2560 scope.go:117] "RemoveContainer" containerID="8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52"
Aug 03 17:03:30.096778 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.096762    2560 scope.go:117] "RemoveContainer" containerID="8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52"
Aug 03 17:03:30.097020 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:03:30.097000    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52\": container with ID starting with 8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52 not found: ID does not exist" containerID="8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52"
Aug 03 17:03:30.097067 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.097028    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52"} err="failed to get container status \"8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52\": rpc error: code = NotFound desc = could not find container \"8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52\": container with ID starting with 8f22f64895d9aa00f87963b1e52d55c0990950a3deab52fed0d6541203217f52 not found: ID does not exist"
Aug 03 17:03:30.097067 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.097044    2560 scope.go:117] "RemoveContainer" containerID="0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9"
Aug 03 17:03:30.105000 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.104970    2560 scope.go:117] "RemoveContainer" containerID="0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9"
Aug 03 17:03:30.105268 ip-10-0-133-47 kubenswrapper[2560]: E0803 17:03:30.105243    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9\": container with ID starting with 0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9 not found: ID does not exist" containerID="0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9"
Aug 03 17:03:30.105334 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.105277    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9"} err="failed to get container status \"0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9\": rpc error: code = NotFound desc = could not find container \"0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9\": container with ID starting with 0a22cc92b5631dcdbfa57cc901d6ff4c4b9a21bcd1e00c1857d7dcb080742fd9 not found: ID does not exist"
Aug 03 17:03:30.106323 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.106304    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"]
Aug 03 17:03:30.115862 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.115823    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7055f0-6dd798596-blx99"]
Aug 03 17:03:30.124980 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.124957    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"]
Aug 03 17:03:30.131120 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.131099    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7055f0-695c6c8f5b-dqkl7"]
Aug 03 17:03:30.147382 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.147362    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"]
Aug 03 17:03:30.154040 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.154015    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7055f0-7b4c4c7b8b-zp6bm"]
Aug 03 17:03:30.261125 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.261083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:30.987424 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.987388    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0118505a-4d26-4cfd-b95a-32e5dbd3927b" path="/var/lib/kubelet/pods/0118505a-4d26-4cfd-b95a-32e5dbd3927b/volumes"
Aug 03 17:03:30.987836 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.987820    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d894c9e-ca69-4931-8184-a792d69d2e6c" path="/var/lib/kubelet/pods/6d894c9e-ca69-4931-8184-a792d69d2e6c/volumes"
Aug 03 17:03:30.988162 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.988149    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c29b03-f7a5-4fe5-9be7-52f28d7dec2b" path="/var/lib/kubelet/pods/79c29b03-f7a5-4fe5-9be7-52f28d7dec2b/volumes"
Aug 03 17:03:30.988537 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:30.988510    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9df64002-fab9-41da-8eb5-033d08ac228e" path="/var/lib/kubelet/pods/9df64002-fab9-41da-8eb5-033d08ac228e/volumes"
Aug 03 17:03:35.876985 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:35.876957    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:35.947958 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:35.947927    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:36.211162 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:36.211068    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:36.489755 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:36.489666    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:43.832332 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:43.832298    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wt9m8_0cfda7ab-dec3-4780-9dc2-1f220a44ca8d/manager/0.log"
Aug 03 17:03:43.974813 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:43.974778    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xzh7x_f544be1b-e7ba-4bdf-a53e-e6323944b47f/manager/0.log"
Aug 03 17:03:44.128453 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:44.128371    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69978859c5-k72h6_96428d5f-8e38-45e4-b12b-eb7d7e6a6853/manager/0.log"
Aug 03 17:03:44.625546 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:44.625498    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-48277_f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb/copy-manifests/0.log"
Aug 03 17:03:44.633628 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:44.633604    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-48277_f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb/manager/0.log"
Aug 03 17:03:45.399077 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:45.399047    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6cd6bc484-t26c5_25b6c0dd-e8d2-4e00-a25e-dcb60c1f8db0/maas-api/0.log"
Aug 03 17:03:45.531124 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:45.531093    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29762910-nqnkz_65f47945-d6d7-47b4-a28b-2c848e239965/cleanup/0.log"
Aug 03 17:03:45.965071 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:45.965040    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-dwv92_6a69665a-9486-4ac5-bcfb-49c8abe964e3/postgres/0.log"
Aug 03 17:03:48.362058 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:48.362026    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2r545_28fb93ca-cb71-43b9-ba03-bee7f777a270/discovery/0.log"
Aug 03 17:03:48.645294 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:48.645214    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-ttdw7_d3a53a64-844f-438c-8759-cebeb5763647/kube-auth-proxy/0.log"
Aug 03 17:03:48.958458 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:48.958356    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bd7sz_f0f4e207-00be-4317-aa53-7b27628a9be5/payload-pre-processing/0.log"
Aug 03 17:03:49.119078 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:49.119039    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lstfn_8e84fafc-9d3d-4122-9b7c-67e622ad5028/payload-processing/0.log"
Aug 03 17:03:49.257165 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:49.257075    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7646c97b6d-frnzn_50306696-a225-4a3f-b23a-5277b7d4563e/router/0.log"
Aug 03 17:03:49.698215 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:49.698182    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-j4pt5_172d98ac-835e-474b-a43a-455aa335c675/main/0.log"
Aug 03 17:03:49.987082 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:49.986992    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-x99mk_92d1c354-a475-41fc-832e-795cdff274d4/main/0.log"
Aug 03 17:03:57.600305 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:57.600217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:03:57.695313 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:03:57.695282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log" limit=1024
Aug 03 17:04:00.614107 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:00.614070    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zkvxk_cd3d71ba-13b7-41e1-a0ab-095fd1f4509c/global-pull-secret-syncer/0.log"
Aug 03 17:04:00.674444 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:00.674409    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fl8dn_bf5d37db-5050-4930-b0a2-2869be2cc445/konnectivity-agent/0.log"
Aug 03 17:04:00.771436 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:00.771400    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-47.ec2.internal_002a780e41fc6d504e461c3d4d84078e/haproxy/0.log"
Aug 03 17:04:01.229549 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:01.229503    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 17:04:01.242252 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:01.242221    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 17:04:01.628513 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:01.628482    2560 scope.go:117] "RemoveContainer" containerID="c85ea1258163590c74856abdf3901bfe1698806df588cd58f0e0f56665420a1f"
Aug 03 17:04:07.479658 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:07.479623    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-p4b6q_c1c80cef-5998-485d-aacb-7d72d6a9d1fc/cluster-monitoring-operator/0.log"
Aug 03 17:04:07.937426 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:07.937394    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p646h_172b4ca6-9e01-4f1e-9386-83ea186d3b56/node-exporter/0.log"
Aug 03 17:04:07.969197 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:07.969170    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p646h_172b4ca6-9e01-4f1e-9386-83ea186d3b56/kube-rbac-proxy/0.log"
Aug 03 17:04:08.005795 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:08.005768    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p646h_172b4ca6-9e01-4f1e-9386-83ea186d3b56/init-textfile/0.log"
Aug 03 17:04:09.888209 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:09.888165    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-wnbm6_3d8c27a8-7bad-47a2-ba46-2ab62d90b198/networking-console-plugin/0.log"
Aug 03 17:04:10.298614 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.298575    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"]
Aug 03 17:04:10.299018 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299000    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0118505a-4d26-4cfd-b95a-32e5dbd3927b" containerName="payload-processing"
Aug 03 17:04:10.299108 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299021    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0118505a-4d26-4cfd-b95a-32e5dbd3927b" containerName="payload-processing"
Aug 03 17:04:10.299108 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299054    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d894c9e-ca69-4931-8184-a792d69d2e6c" containerName="payload-pre-processing"
Aug 03 17:04:10.299108 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299064    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d894c9e-ca69-4931-8184-a792d69d2e6c" containerName="payload-pre-processing"
Aug 03 17:04:10.299108 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299082    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df64002-fab9-41da-8eb5-033d08ac228e" containerName="maas-api"
Aug 03 17:04:10.299108 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299091    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df64002-fab9-41da-8eb5-033d08ac228e" containerName="maas-api"
Aug 03 17:04:10.299108 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299107    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c29b03-f7a5-4fe5-9be7-52f28d7dec2b" containerName="revoke-keys"
Aug 03 17:04:10.299407 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299116    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c29b03-f7a5-4fe5-9be7-52f28d7dec2b" containerName="revoke-keys"
Aug 03 17:04:10.299407 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299208    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df64002-fab9-41da-8eb5-033d08ac228e" containerName="maas-api"
Aug 03 17:04:10.299407 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299221    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d894c9e-ca69-4931-8184-a792d69d2e6c" containerName="payload-pre-processing"
Aug 03 17:04:10.299407 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299231    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0118505a-4d26-4cfd-b95a-32e5dbd3927b" containerName="payload-processing"
Aug 03 17:04:10.299407 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.299243    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c29b03-f7a5-4fe5-9be7-52f28d7dec2b" containerName="revoke-keys"
Aug 03 17:04:10.304238 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.304217    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.307081 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.307063    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xpjhq\"/\"default-dockercfg-dl5pw\""
Aug 03 17:04:10.308151 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.308131    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpjhq\"/\"openshift-service-ca.crt\""
Aug 03 17:04:10.308270 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.308216    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpjhq\"/\"kube-root-ca.crt\""
Aug 03 17:04:10.311430 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.311405    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"]
Aug 03 17:04:10.393345 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.393311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-proc\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.393345 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.393349    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-podres\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.393641 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.393370    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-lib-modules\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.393641 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.393422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bpvh\" (UniqueName: \"kubernetes.io/projected/5722f30a-a89a-4c52-85b5-fcba8116b398-kube-api-access-4bpvh\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.393641 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.393521    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-sys\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494498 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-proc\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494498 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494499    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-podres\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-lib-modules\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bpvh\" (UniqueName: \"kubernetes.io/projected/5722f30a-a89a-4c52-85b5-fcba8116b398-kube-api-access-4bpvh\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-sys\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494654    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-podres\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494699    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-sys\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494708    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-lib-modules\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.494727 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.494721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5722f30a-a89a-4c52-85b5-fcba8116b398-proc\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.505613 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.505588    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bpvh\" (UniqueName: \"kubernetes.io/projected/5722f30a-a89a-4c52-85b5-fcba8116b398-kube-api-access-4bpvh\") pod \"perf-node-gather-daemonset-7wqz8\" (UID: \"5722f30a-a89a-4c52-85b5-fcba8116b398\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.571504 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.571417    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/1.log"
Aug 03 17:04:10.579302 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.579272    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-4xfp9_3181b3d5-81de-4ea5-984c-d83d965219cc/console-operator/2.log"
Aug 03 17:04:10.614756 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.614726    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:10.747645 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:10.747619    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"]
Aug 03 17:04:10.749810 ip-10-0-133-47 kubenswrapper[2560]: W0803 17:04:10.749787    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5722f30a_a89a_4c52_85b5_fcba8116b398.slice/crio-beb9feee6027ea373506e2bc8ed12ecbca52f89a507d09ef4db2b120a113a8fe WatchSource:0}: Error finding container beb9feee6027ea373506e2bc8ed12ecbca52f89a507d09ef4db2b120a113a8fe: Status 404 returned error can't find the container with id beb9feee6027ea373506e2bc8ed12ecbca52f89a507d09ef4db2b120a113a8fe
Aug 03 17:04:11.205258 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:11.205227    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-wx647_92a1c9e2-aad4-47fb-8b4f-4ff3bc520e17/download-server/0.log"
Aug 03 17:04:11.285599 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:11.285567    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8" event={"ID":"5722f30a-a89a-4c52-85b5-fcba8116b398","Type":"ContainerStarted","Data":"8f9e920b061d971edec5c87a59555c6b92ae3ddd1eb6608415cb7af8cdcaadbb"}
Aug 03 17:04:11.285788 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:11.285633    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8" event={"ID":"5722f30a-a89a-4c52-85b5-fcba8116b398","Type":"ContainerStarted","Data":"beb9feee6027ea373506e2bc8ed12ecbca52f89a507d09ef4db2b120a113a8fe"}
Aug 03 17:04:11.285788 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:11.285665    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:11.305301 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:11.305248    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8" podStartSLOduration=1.305231655 podStartE2EDuration="1.305231655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 17:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:04:11.304270465 +0000 UTC m=+3610.853556782" watchObservedRunningTime="2026-08-03 17:04:11.305231655 +0000 UTC m=+3610.854518013"
Aug 03 17:04:11.882646 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:11.882612    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-5tx54_15beb5d1-c300-43b2-ba75-232cc1167840/volume-data-source-validator/0.log"
Aug 03 17:04:13.071792 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:13.071760    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cltww_7002810c-9381-4adb-95cb-f90d71adb444/dns/0.log"
Aug 03 17:04:13.121996 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:13.121970    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cltww_7002810c-9381-4adb-95cb-f90d71adb444/kube-rbac-proxy/0.log"
Aug 03 17:04:13.322491 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:13.322412    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzdx8_17975002-99dd-4015-a156-db842277c1ad/dns-node-resolver/0.log"
Aug 03 17:04:14.029845 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:14.029816    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xkv75_0ecec175-b462-4ce4-a083-b2192c1b83ab/node-ca/0.log"
Aug 03 17:04:15.413417 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:15.413381    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2r545_28fb93ca-cb71-43b9-ba03-bee7f777a270/discovery/0.log"
Aug 03 17:04:15.469690 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:15.469661    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-ttdw7_d3a53a64-844f-438c-8759-cebeb5763647/kube-auth-proxy/0.log"
Aug 03 17:04:15.557770 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:15.557735    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bd7sz_f0f4e207-00be-4317-aa53-7b27628a9be5/payload-pre-processing/0.log"
Aug 03 17:04:15.602463 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:15.602434    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lstfn_8e84fafc-9d3d-4122-9b7c-67e622ad5028/payload-processing/0.log"
Aug 03 17:04:15.630267 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:15.630241    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7646c97b6d-frnzn_50306696-a225-4a3f-b23a-5277b7d4563e/router/0.log"
Aug 03 17:04:16.490098 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:16.490066    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l7gnr_957464d8-2258-4cce-acf5-0d5fecccebd9/serve-healthcheck-canary/0.log"
Aug 03 17:04:17.040650 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:17.040614    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-rt655_12a79723-b930-4937-9c84-ac2f2b437efb/insights-operator/0.log"
Aug 03 17:04:17.041753 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:17.041730    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-rt655_12a79723-b930-4937-9c84-ac2f2b437efb/insights-operator/1.log"
Aug 03 17:04:17.188123 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:17.188093    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ld9nn_10aead5f-7f93-4830-8756-f30d0e554e5c/kube-rbac-proxy/0.log"
Aug 03 17:04:17.213732 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:17.213699    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ld9nn_10aead5f-7f93-4830-8756-f30d0e554e5c/exporter/0.log"
Aug 03 17:04:17.238737 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:17.238711    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ld9nn_10aead5f-7f93-4830-8756-f30d0e554e5c/extractor/0.log"
Aug 03 17:04:17.301597 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:17.301499    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-7wqz8"
Aug 03 17:04:19.575000 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:19.574971    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wt9m8_0cfda7ab-dec3-4780-9dc2-1f220a44ca8d/manager/0.log"
Aug 03 17:04:19.801260 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:19.801236    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xzh7x_f544be1b-e7ba-4bdf-a53e-e6323944b47f/manager/0.log"
Aug 03 17:04:20.130421 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.130328    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69978859c5-k72h6_96428d5f-8e38-45e4-b12b-eb7d7e6a6853/manager/0.log"
Aug 03 17:04:20.247190 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.247161    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-48277_f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb/manager/0.log"
Aug 03 17:04:20.273767 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.273743    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-48277_f71ed6ff-8851-4538-ba1c-cdbd36fcf4bb/copy-manifests/0.log"
Aug 03 17:04:20.852694 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.852660    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2r545_28fb93ca-cb71-43b9-ba03-bee7f777a270/discovery/0.log"
Aug 03 17:04:20.876942 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.876913    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-ttdw7_d3a53a64-844f-438c-8759-cebeb5763647/kube-auth-proxy/0.log"
Aug 03 17:04:20.933884 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.933855    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bd7sz_f0f4e207-00be-4317-aa53-7b27628a9be5/payload-pre-processing/0.log"
Aug 03 17:04:20.964079 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.964041    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lstfn_8e84fafc-9d3d-4122-9b7c-67e622ad5028/payload-processing/0.log"
Aug 03 17:04:20.983969 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:20.983930    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7646c97b6d-frnzn_50306696-a225-4a3f-b23a-5277b7d4563e/router/0.log"
Aug 03 17:04:22.648646 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:22.648614    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-2r545_28fb93ca-cb71-43b9-ba03-bee7f777a270/discovery/0.log"
Aug 03 17:04:22.678617 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:22.678589    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-845c84c75b-ttdw7_d3a53a64-844f-438c-8759-cebeb5763647/kube-auth-proxy/0.log"
Aug 03 17:04:22.738942 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:22.738905    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bd7sz_f0f4e207-00be-4317-aa53-7b27628a9be5/payload-pre-processing/0.log"
Aug 03 17:04:22.777645 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:22.777600    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-lstfn_8e84fafc-9d3d-4122-9b7c-67e622ad5028/payload-processing/0.log"
Aug 03 17:04:22.797686 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:22.797653    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7646c97b6d-frnzn_50306696-a225-4a3f-b23a-5277b7d4563e/router/0.log"
Aug 03 17:04:26.494079 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:26.493926    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-ddx7k_d258a910-019a-4e1e-8a08-7e63343e0eba/migrator/0.log"
Aug 03 17:04:26.520569 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:26.520517    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-ddx7k_d258a910-019a-4e1e-8a08-7e63343e0eba/graceful-termination/0.log"
Aug 03 17:04:26.911293 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:26.911258    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-9qzsb_0c729d6b-549e-4975-87be-65b77ffd61e4/kube-storage-version-migrator-operator/1.log"
Aug 03 17:04:26.912852 ip-10-0-133-47 kubenswrapper[2560]: I0803 17:04:26.912816    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-9qzsb_0c729d6b-549e-4975-87be-65b77ffd61e4/kube-storage-version-migrator-operator/0.log"